NomenAK

We built jev-tools to give AI coding agents a better way to verify their work Hey everyone! We've been working on jev-tools, an open-source toolkit designed to improve how AI coding agents explore codebases, gather evidence, and make decisions. GitHub: NomenAK/jev-tools One problem we've been thinking about is how coding agents can confidently make incorrect assumptions about a codebase. An agent might claim that a feature is implemented, overlook a relevant test, misunderstand how two modules interact, or miss documentation that no longer matches the code. Simply asking the same model to double-check its answer doesn't necessarily solve the problem. So we're exploring a different approach: give coding agents dedicated tools to investigate their claims using actual repository evidence. What we've built jev-tools currently provides six tools for different parts of the development workflow: - jev_ask: Ask focused questions using evidence gathered from multiple sources. - jev_ask_files: Evaluate a question independently across multiple files. - jev_find_files: Find files relevant to a particular behavior or development goal. - jev_locate_in_file: Locate relevant information within source files. - jev_check_diff: Examine changes for potential inconsistencies and issues. - jev_select_tests: Identify existing tests relevant to modified code. We've also built MCP support so compatible coding agents can access these capabilities. Another interesting part is documentation consistency checking. Our integrations can check whether documentation may have become outdated after code changes. The idea behind the project We see evidence collection and semantic judgment as two distinct responsibilities. The agent gathers relevant information from the repository. A separate judgment mechanism evaluates a specific question using that evidence. The agent can then use the result to decide what to investigate or do next. This doesn't eliminate hallucinations, and we don't consider a positive judgment proof of correctness. The evidence may be incomplete, and the judgment itself may be wrong. Our goal is to explore whether giving agents more structured ways to investigate uncertainty can improve their reliability in real development workflows. We'd love your feedback We're particularly interested in hearing from people building AI agents, coding assistants, and agent harnesses. A few questions we're exploring: 1. How are you currently verifying the claims made by AI coding agents? 2. Would dedicated evidence-gathering and judgment tools improve your existing agent workflows? 3. How would you benchmark whether these tools actually reduce incorrect conclusions? 4. Would you prefer a locally hosted judgment model over an external API? 5. What failure cases should we investigate next? We'd appreciate honest feedback, including criticism of the architecture, alternative approaches, and ideas for improving the project. Repository: NomenAK/jev-tools We're building this in the open and would love to hear what the community thinks.

We built jev-tools to give AI coding agents a better way to verify their work Hey everyone! We've been working on jev-tools, an open-source toolkit designed to improve how AI coding agents explore codebases, gather evidence, and make decisions. GitHub: NomenAK/jev-tools One problem we've been thinking about is how coding agents can confidently make incorrect assumptions about a codebase. An agent might claim that a feature is implemented, overlook a relevant test, misunderstand how two modules interact, or miss documentation that no longer matches the code. Simply asking the same model to double-check its answer doesn't necessarily solve the problem. So we're exploring a different approach: give coding agents dedicated tools to investigate their claims using actual repository evidence. What we've built jev-tools currently provides six tools for different parts of the development workflow: - jev_ask: Ask focused questions using evidence gathered from multiple sources. - jev_ask_files: Evaluate a question independently across multiple files. - jev_find_files: Find files relevant to a particular behavior or development goal. - jev_locate_in_file: Locate relevant information within source files. - jev_check_diff: Examine changes for potential inconsistencies and issues. - jev_select_tests: Identify existing tests relevant to modified code. We've also built MCP support so compatible coding agents can access these capabilities. Another interesting part is documentation consistency checking. Our integrations can check whether documentation may have become outdated after code changes. The idea behind the project We see evidence collection and semantic judgment as two distinct responsibilities. The agent gathers relevant information from the repository. A separate judgment mechanism evaluates a specific question using that evidence. The agent can then use the result to decide what to investigate or do next. This doesn't eliminate hallucinations, and we don't consider a positive judgment proof of correctness. The evidence may be incomplete, and the judgment itself may be wrong. Our goal is to explore whether giving agents more structured ways to investigate uncertainty can improve their reliability in real development workflows. We'd love your feedback We're particularly interested in hearing from people building AI agents, coding assistants, and agent harnesses. A few questions we're exploring: 1. How are you currently verifying the claims made by AI coding agents? 2. Would dedicated evidence-gathering and judgment tools improve your existing agent workflows? 3. How would you benchmark whether these tools actually reduce incorrect conclusions? 4. Would you prefer a locally hosted judgment model over an external API? 5. What failure cases should we investigate next? We'd appreciate honest feedback, including criticism of the architecture, alternative approaches, and ideas for improving the project. Repository: NomenAK/jev-tools We're building this in the open and would love to hear what the community thinks.