UKGovernmentBEIS

Inspect AI is a Python framework for LLM evaluations, built by the UK AI Security Institute. It ships with over 200 pre-built evals you can run on any model, plus tool use, multi-turn dialog and model-graded scoring. UKGovernmentBEIS/inspect_ai t.co

6 Frameworks for Evaluating Agent Tool Use An AI agent can produce a convincing final answer while using the wrong tools, passing incorrect arguments, or performing unnecessary actions along the way. Consider an agent responsible for scheduling meetings. It may successfully create a calendar event but invite the wrong participants, select an incorrect time zone, or fail to check availability before confirming the booking. From the user's perspective, the task appears complete. From an engineering perspective, the execution may be incorrect. This is why evaluating agent tool use requires more than measuring final response quality. For ML engineers and technical evaluators, six frameworks worth examining are Inspect AI, DeepEval, LangSmith, Braintrust, Promptfoo, and AgentBench. These tools serve different evaluation requirements. Some support detailed execution traces and trajectory analysis. Others provide automated assertions, controlled environments, or benchmarks for agent capabilities. This article explains how each framework approaches agent tool-use evaluation, where it fits, and what engineering teams should measure before deploying tool-using agents in production. Why Agent Tool Use Needs Its Own Evaluation Strategy Traditional language model evaluations often focus on the quality of generated text. Tool-using agents introduce additional behaviors. An agent may need to: 1. Interpret the user's request. 2. Decide whether a tool is necessary. 3. Select the appropriate tool. 4. Construct valid arguments. 5. Execute the tool. 6. Interpret the result. 7. Decide whether another action is needed. 8. Complete the task. 9. Report the outcome accurately. A failure at any stage can affect the final result. For example, a customer support agent might select the correct refund API but pass an incorrect transaction identifier. Alternatively, it might execute a refund before checking whether the customer meets the refund policy. These are different failure modes. The first concerns argument correctness. The second concerns workflow compliance and authorization. A reliable evaluation process must distinguish between them. What Should You Measure in Agent Tool Use? Before selecting an evaluation framework, define the behaviors that matter for your application. Tool Selection Accuracy Did the agent choose the appropriate tool for the task? For example, an agent should use a database query tool to retrieve current inventory rather than generate an answer from memory. Argument Correctness Were the tool arguments valid and consistent with the user's request? A correctly selected tool can still produce an incorrect result when supplied with wrong parameters. Tool Execution Success Did the tool call execute successfully? This includes handling authentication failures, invalid inputs, rate limits, and unavailable services. Execution Order Did the agent perform actions in the required sequence? For example, an agent may need to verify an order before issuing a refund. Task Completion Did the agent achieve the intended outcome? Task completion should be measured independently from the agent's own claim of success. Unnecessary Tool Calls Did the agent execute tools that were not needed? Unnecessary calls can increase latency, cost, and operational risk. Recovery From Failures Can the agent recover from tool errors without repeating consequential actions? Authorization and Safety Did the agent stay within its permitted scope? This is particularly important when tools can modify records, transfer information, or trigger external actions. These dimensions form the basis of a practical evaluation strategy. --- 1. Inspect AI: Controlled Tool-Use and Agent Evaluations Official documentation: inspect.aisi.org.uk GitHub: UKGovernmentBEIS/inspect_ai Inspect AI is an open-source evaluation framework developed for structured assessment of language models and agentic systems. It provides components for defining evaluation tasks, supplying tools, executing model interactions, and scoring outcomes. Its support for sandboxed environments makes it particularly relevant for agents that execute code or interact with external systems. Key Features - Custom evaluation tasks. - Agent and model evaluation. - Tool integration. - Multi-turn execution. - Sandboxed environments. - Custom scoring functions. - Evaluation datasets. - Execution logging. - Model comparison. Where Inspect AI Works Well Consider an AI coding agent that must modify a repository and run tests. The agent may need to inspect files, edit code, execute commands, and verify the result. Evaluating only the final message would be insufficient. The engineering team needs to know whether the agent actually changed the correct files and whether the resulting code passes tests. Inspect AI can support this type of evaluation through controlled execution environments and task-specific scorers. Example: Defining a Simple Evaluation Task Install Inspect AI: pip install inspect-ai A simplified evaluation task can be defined as follows: from inspect_ai import Task, task from inspect_ai.dataset import Sample from inspect_ai.scorer import exact from inspect_ai.solver import generate @task def simple_tool_reasoning(): return Task( dataset=[ Sample( input=( "Which tool should be used " "to retrieve current inventory?" ), target="inventory_lookup" ) ], solver=generate(), scorer=exact() ) This example tests the model's textual tool choice rather than executing a tool. A real agent evaluation would provide tool definitions, record tool calls, and use a scorer that checks execution outcomes. Advantages Inspect AI supports reproducible evaluation tasks and controlled execution. Its sandboxing capabilities are valuable when evaluating agents that execute code or interact with environments. Limitations Inspect AI requires engineers to define appropriate tasks, tools, and scoring logic. It is not a complete replacement for production tracing or operational monitoring. Best Suited For - Coding agents. - Tool-using research agents. - Controlled execution benchmarks. - Multi-step agent evaluation. - Sandboxed testing. - Reproducible model comparisons. Technical verdict: Inspect AI is a strong option when agent evaluation requires controlled environments and explicit task-level scoring. --- 2. DeepEval: Automated Agent Tool-Use Testing Official documentation: deepeval.com GitHub: confident-ai/deepeval DeepEval is an open-source evaluation framework for testing LLM applications. It provides metrics for assessing agent behavior, including tool selection, tool execution, and task completion. Its Python-oriented testing approach makes it useful for engineering teams that want to integrate agent evaluations into development workflows. Key Features - Tool correctness evaluation. - Agent task completion metrics. - Tool call recording. - Custom evaluation criteria. - LLM-based metrics. - Test case management. - Pytest-style workflows. - CI integration. Where DeepEval Works Well Consider an AI agent that processes customer support requests. The agent may have access to: - Customer lookup. - Order history. - Refund eligibility. - Refund execution. - Ticket creation. A test case might require the agent to retrieve the customer record, check eligibility, and create a support ticket. DeepEval can help assess whether the agent selected appropriate tools and completed the intended task. Example: Evaluating Tool Correctness from deepeval import evaluate from deepeval.metrics import ToolCorrectnessMetric from deepeval.test_case import ( LLMTestCase, ToolCall ) test_case = LLMTestCase( input="Find the status of order 12345.", actual_output="Order 12345 has shipped.", tools_called=[ ToolCall( name="order_lookup", input_parameters={ "order_id": "12345" } ) ], expected_tools=[ ToolCall( name="order_lookup", input_parameters={ "order_id": "12345" } ) ] ) metric = ToolCorrectnessMetric() evaluate( test_cases=[test_case], metrics=[metric] ) This example checks recorded tool calls against expected tool calls. It does not independently verify whether the order actually shipped. That requires a trusted test environment or an additional outcome assertion. Advantages DeepEval allows teams to define tool-use tests in Python and run them alongside other automated checks. This can make regression testing easier to incorporate into CI pipelines. Limitations Tool correctness metrics depend on how expected behavior is defined. Some tasks have multiple valid tool sequences. A rigid expected sequence may incorrectly penalize an agent that completes the task using another valid approach. Best Suited For - Python-based agent applications. - Tool selection testing. - Tool argument validation. - CI regression testing. - Agent task completion evaluation. Technical verdict: DeepEval is a practical option for teams that want automated agent tool-use tests integrated into their software development process. --- 3. LangSmith: Trajectory Evaluation and Execution Tracing Official documentation: LangSmith Evaluation Documentation Official website: LangSmith LangSmith is a platform for tracing, evaluating, and monitoring LLM applications, including agents built with LangChain, LangGraph, and other frameworks. Its primary advantage for tool-use evaluation is visibility into the agent's execution trajectory. Instead of examining only the final answer, engineers can inspect which tools were called, their inputs and outputs, and the sequence of decisions. Licensing distinction: LangSmith is a commercial platform with self-hosting options, not an open-source evaluation framework. It is included because it addresses the same engineering problem as the other tools in this shortlist. Key Features - Agent execution tracing. - Tool-call inspection. - Trajectory evaluation. - Offline evaluation datasets. - Online evaluation. - Custom evaluators. - Human feedback. - Experiment comparison. - Integration with LangGraph and other agent frameworks. Where LangSmith Works Well Consider an agent responsible for resolving customer support requests. The expected workflow is: 1. Retrieve the customer's record. 2. Identify the relevant order. 3. Check refund eligibility. 4. Request approval when necessary. 5. Execute an authorized refund. 6. Report the outcome. The agent may complete the refund but skip the approval step. A final-answer evaluation might incorrectly classify the interaction as successful. Trajectory evaluation allows engineers to inspect the sequence of tool calls and identify the missing approval. How to Evaluate Tool Trajectories LangSmith supports approaches such as comparing an actual trajectory against a reference trajectory or using an evaluator to assess the execution path. For example, a test case could specify: { "task": "Refund an eligible order", "expected_tools": [ "customer_lookup", "order_lookup", "refund_eligibility", "approval_check", "refund_execution" ] } The evaluator can compare the recorded tool sequence with the expected workflow. However, exact sequence matching is not appropriate for every task. An agent may complete some workflows correctly through multiple valid paths. In those situations, evaluators should check required actions, prohibited actions, and final outcomes rather than demanding one exact sequence. Advantages LangSmith is useful for debugging complex agent workflows and evaluating behavior against recorded execution traces. Its integration with LangGraph makes it relevant for teams already using that ecosystem. Limitations LangSmith is not an open-source framework. Teams should assess data handling, pricing, deployment requirements, and vendor dependencies before adoption. Tracing also introduces operational considerations because tool inputs and outputs may contain sensitive information. Best Suited For - LangGraph-based agents. - Complex multistep workflows. - Trajectory evaluation. - Production agent debugging. - Tool-call regression testing. - Human review of agent execution. Technical verdict: LangSmith is worth evaluating when trace visibility and trajectory analysis are priorities, particularly for teams using LangGraph. --- 4. Braintrust: Dataset-Driven Evaluation of Agent Workflows Official website: Braintrust Documentation: Braintrust Documentation Braintrust is an evaluation and observability platform for AI applications. It supports datasets, experiments, custom scoring functions, and tracing. For agent evaluation, its main value is the ability to compare application behavior against defined test cases while retaining execution context. Licensing distinction: Braintrust is a commercial platform with open-source SDKs and supporting components. The hosted evaluation platform itself should not be treated as fully open source. Key Features - Evaluation datasets. - Custom scoring functions. - Experiment tracking. - Agent execution tracing. - Human feedback. - Model comparison. - Production monitoring. - Integration with application code. - Regression evaluation. Where Braintrust Works Well Consider a procurement agent that can search supplier records, retrieve contract information, and prepare purchase requests. The agent should select appropriate tools and follow procurement rules. A test dataset might include: - Requests from authorized employees. - Requests exceeding approval limits. - Suppliers with expired contracts. - Missing purchase order information. - Tool failures. - Attempts to bypass purchasing policies. The evaluation process should determine whether the agent handles each scenario correctly. Example: A Custom Tool-Use Scorer A custom Python scorer can inspect recorded tool calls. def required_tool_score(output, expected): actual_tools = { call["name"] for call in output["tool_calls"] } required_tools = set( expected["required_tools"] ) return float( required_tools.issubset(actual_tools) ) This function returns 1 when all required tools appear in the recorded execution and 0 otherwise. It is a standalone scoring function, not a complete Braintrust SDK example. A production evaluator should also check arguments, ordering constraints, prohibited actions, and actual task outcomes. Advantages Braintrust provides a dataset-driven approach to evaluating application behavior. Its experiment and tracing capabilities can help teams compare agent implementations. Limitations The evaluation platform is commercial. Teams should assess deployment requirements, data governance, pricing, and the degree of integration with their existing engineering systems. Best Suited For - Dataset-driven agent evaluation. - Custom tool-use scoring. - Experiment comparison. - Production trace analysis. - Regression testing. - Teams requiring a shared evaluation platform. Technical verdict: Braintrust is a useful option for organizations that want centralized experiments, custom scoring, and agent observability, provided its commercial model fits their requirements. --- 5. Promptfoo: Tool-Call Assertions and Agent Regression Testing Official documentation: Promptfoo Documentation GitHub: Promptfoo Repository Promptfoo is an open-source framework for evaluating LLM applications and agents through configurable test cases and assertions. It is particularly useful when engineering teams want to validate tool calls as part of automated regression testing. Promptfoo supports tool-call validation, trajectory assertions, tracing, and security testing. Its documentation describes ways to inspect tool names, arguments, execution order, and errors using OpenTelemetry traces. Key Features - Tool-call schema validation. - Tool selection assertions. - Tool argument matching. - Tool execution sequence checks. - Trajectory evaluation. - Custom assertions. - Model and prompt comparison. - CI integration. - Agent red teaming. - OpenTelemetry tracing. Where Promptfoo Works Well Consider an agent that retrieves order information and answers customer questions. The agent should use search_orders with the correct order identifier. A test should verify that the agent actually called the tool rather than inventing an order status. It should also verify that the tool received the correct arguments. Example: Testing Tool Calls The following YAML illustrates trajectory assertions for an instrumented agent: description: Order lookup agent evaluation tracing: enabled: true tests: - vars: order_id: "12345" assert: - type: trajectory:tool-used value: search_orders - type: trajectory:tool-args-match value: name: search_orders args: order_id: "12345" These assertions require the evaluated application to emit compatible tool-call traces. A complete runnable configuration also needs a provider and the application under test. Testing Execution Order For workflows where order matters, Promptfoo supports trajectory sequence assertions. For example, an agent should verify an order before processing a refund. assert: - type: trajectory:tool-sequence value: steps: - verify_order - process_refund This checks the recorded sequence. It does not independently prove that the refund was authorized or correctly processed. For those requirements, engineers should add assertions against trusted application state. Advantages Promptfoo is useful for automated regression testing because many tool-use requirements can be expressed as deterministic assertions. Its tracing capabilities also help engineers investigate why a test failed. Limitations Trajectory assertions depend on complete and correctly instrumented traces. A tool call that is not captured cannot be evaluated reliably. Strict sequence assertions can also be inappropriate for workflows that permit multiple valid execution paths. Best Suited For - Tool-call regression testing. - API argument validation. - CI pipelines. - Agent security testing. - Execution sequence verification. - Prompt and model comparisons. Technical verdict: Promptfoo is a strong choice for engineering teams that need repeatable tool-use assertions and automated agent regression testing. --- 6. AgentBench: Benchmarking Agents in Interactive Environments Official repository: AgentBench GitHub Repository Research paper: AgentBench: Evaluating LLMs as Agents AgentBench is an open-source benchmark designed to evaluate language models as agents operating in interactive environments. Unlike application-specific testing frameworks, AgentBench focuses on assessing general agent capabilities across predefined environments. The original research introduced eight environments to evaluate reasoning, decision-making, and interaction capabilities. The paper was published at ICLR 2024. Key Features - Interactive evaluation environments. - Multistep agent tasks. - Environment-based success measurement. - Model comparison. - Agent reasoning assessment. - Decision-making evaluation. - Reproducible benchmark tasks. - Research-oriented evaluation. Where AgentBench Works Well Consider a research team comparing several language models for agentic applications. The team wants to understand how models perform when they must interact with environments rather than simply answer questions. AgentBench provides predefined tasks that can help reveal differences in interactive decision-making. This is useful during model selection or research. Why Interactive Evaluation Matters A model may answer a question about database operations correctly while failing to perform those operations in a simulated environment. Interactive tasks introduce additional requirements: 1. Understand the current environment. 2. Select an action. 3. Execute the action. 4. Interpret feedback. 5. Adjust the next action. 6. Continue until the objective is achieved or the task fails. These capabilities are difficult to assess using static question-answer benchmarks. Advantages AgentBench provides a standardized approach to evaluating agent behavior across interactive tasks. It can support research into model capabilities and limitations. Limitations AgentBench is a benchmark suite rather than a complete production evaluation platform. Its environments may not reflect the exact tools, permissions, workflows, or business requirements of an enterprise application. A model that performs well on AgentBench may still fail on a company's custom API workflows. Teams should therefore supplement benchmark results with application-specific evaluations. Best Suited For - Agent model benchmarking. - Interactive decision-making research. - Comparing language models. - Evaluating multistep behavior. - Research-oriented agent testing. Technical verdict: AgentBench is useful for assessing general agent capabilities, but production teams still need tests based on their own tools and workflows. --- Which Framework Should You Choose? The six frameworks address different evaluation requirements. They should not be treated as interchangeable products. Best for Controlled Agent Execution Inspect AI is a strong candidate when agents must operate inside controlled environments. It is particularly relevant for coding agents, sandboxed execution, and reproducible research evaluations. Best for Python-Based Tool-Use Testing DeepEval provides tool correctness and argument correctness metrics alongside trajectory-level evaluation. It is useful when agent tests should run as part of the Python development workflow. Best for Trajectory Analysis LangSmith is relevant when engineers need detailed execution traces and trajectory evaluation. It is especially useful for applications built with LangGraph. Best for Centralized Evaluation Experiments Braintrust supports dataset-driven experiments, custom scoring, and trace analysis. It is worth evaluating when multiple engineering teams need a shared evaluation environment. Best for CI Regression Testing Promptfoo provides deterministic assertions for tool selection, arguments, and execution order. It is useful when agent behavior must be checked automatically before deployment. Best for Agent Benchmarking AgentBench is appropriate for research-oriented comparisons of agent capabilities across interactive environments. It is not a replacement for production application testing. How to Build a Reliable Agent Tool-Use Evaluation Pipeline A framework provides testing infrastructure. It does not automatically determine whether an agent behaves correctly. That requires a structured evaluation process. Step 1: Define the Agent's Responsibilities Document the tasks the agent is permitted to perform. For example, a customer support agent may be allowed to: - Retrieve customer information. - Check order status. - Review refund eligibility. - Create support tickets. - Request approval for refunds. It may not be permitted to execute certain financial actions without additional authorization. These rules should become testable requirements. Step 2: Define the Available Tools For each tool, document: - Tool name. - Purpose. - Input schema. - Output schema. - Required permissions. - Side effects. - Failure conditions. For example: { "name": "order_lookup", "description": "Retrieve an order by ID", "parameters": { "type": "object", "properties": { "order_id": { "type": "string" } }, "required": ["order_id"] } } This definition establishes the expected input structure. The application should also validate the arguments before executing the operation. Step 3: Create Representative Evaluation Scenarios A useful evaluation dataset should include successful and unsuccessful scenarios. Examples include: Normal operation: The agent retrieves an order using the correct identifier. Missing information: The user requests an order update without providing an order number. Ambiguous request: The user has several orders matching the description. Tool failure: The order lookup API returns an error. Authorization failure: The user requests information about another customer's order. Workflow violation: The agent attempts a refund without checking eligibility. Repeated execution: The agent retries an operation that has already succeeded. Prompt injection: An external document attempts to instruct the agent to bypass its rules. These scenarios reveal different failure modes. Step 4: Record the Execution Trajectory Each agent run should produce enough information to reconstruct the execution. A useful trace includes: - Test case identifier. - User request. - Model and prompt version. - Tool calls. - Tool arguments. - Tool responses. - Execution timestamps. - Errors. - Final response. - Final application state. Sensitive information should be redacted or excluded according to the application's data-handling requirements. Step 5: Evaluate Tool Selection Compare actual tool calls with expected behavior. For tasks with a single valid tool, exact matching may be appropriate. For more flexible workflows, evaluate whether the selected tools were permissible and sufficient. Step 6: Evaluate Arguments Check whether tool arguments satisfy both the schema and the user's actual intent. Schema validation alone is insufficient. For example, an order ID can be a valid string while referring to the wrong order. Step 7: Evaluate Execution Order Identify required dependencies between actions. For example: Retrieve Order | v Check Eligibility | v Request Approval | v Execute Refund The evaluation should detect prohibited transitions. Not every workflow requires exact sequence matching. Independent read-only operations may be executed in different orders without affecting correctness. Step 8: Verify the Final Outcome Do not rely exclusively on the agent's final message. If the agent claims to have created a ticket, verify that the ticket exists. If it claims to have updated a record, inspect the resulting application state. For consequential actions, outcome verification should use a trusted source independent of the agent's generated response. Step 9: Measure Efficiency Track: - Tool calls per completed task. - Redundant tool calls. - Failed tool calls. - Model inference requests. - Total execution latency. - Cost per successful task. Efficiency should be evaluated only after correctness and safety requirements are satisfied. Step 10: Run Regression Tests Repeat evaluations when the team changes: - Language models. - System prompts. - Tool definitions. - API integrations. - Agent orchestration. - Authorization rules. - Retry behavior. Maintain a versioned evaluation dataset so results remain comparable. Example: A Lightweight Tool-Use Evaluator in Python Before adopting a larger evaluation platform, engineers can implement basic deterministic checks. The following example evaluates required tool calls, argument correctness, and prohibited actions. def evaluate_tool_use( actual_calls, required_calls, prohibited_tools ): results = { "required_calls_present": True, "arguments_correct": True, "no_prohibited_tools": True } actual_tool_names = [ call["name"] for call in actual_calls ] for expected in required_calls: if expected["name"] not in actual_tool_names: results["required_calls_present"] = False matching_calls = [ call for call in actual_calls if call["name"] == expected["name"] ] if not any( call.get("arguments") == expected["arguments"] for call in matching_calls ): results["arguments_correct"] = False for call in actual_calls: if call["name"] in prohibited_tools: results["no_prohibited_tools"] = False return results actual_calls = [ { "name": "order_lookup", "arguments": { "order_id": "12345" } }, { "name": "refund_eligibility", "arguments": { "order_id": "12345" } } ] required_calls = [ { "name": "order_lookup", "arguments": { "order_id": "12345" } }, { "name": "refund_eligibility", "arguments": { "order_id": "12345" } } ] prohibited_tools = { "refund_execution" } results = evaluate_tool_use( actual_calls, required_calls, prohibited_tools ) print(results) Expected output: { "required_calls_present": True, "arguments_correct": True, "no_prohibited_tools": True } What This Evaluator Measures The example checks whether: 1. Required tools were called. 2. Expected arguments were supplied. 3. Prohibited tools were avoided. What It Does Not Measure It does not determine whether: - Tools executed successfully. - Calls occurred in the correct order. - The agent completed the task. - The returned data was accurate. - The agent handled errors correctly. - The final application state was valid. Those requirements need additional assertions. For example, a refund workflow should verify that an authorized refund was created exactly once. A production evaluator should also account for repeated calls and workflows with multiple valid execution paths. Metrics for Agent Tool-Use Evaluation A production evaluation process should measure several dimensions independently. Tool Selection Accuracy Measures whether the agent selected the expected tools. For tasks with a single expected tool: Tool selection accuracy = Correct tool selections ÷ Total evaluated selections Argument Accuracy Measures whether tool arguments match the expected values. Argument accuracy = Correctly parameterized calls ÷ Evaluated calls Argument correctness should include semantic checks where necessary. Task Completion Rate Measures the percentage of tasks successfully completed. Task completion rate = Successfully completed tasks ÷ Total attempted tasks Success should be verified through trusted outcomes. Workflow Compliance Rate Measures how often the agent follows mandatory workflow requirements. For example, a refund agent should check eligibility before execution. Unauthorized Action Rate Measures the proportion of evaluated runs containing prohibited actions. This metric should be reported separately from overall task success. Tool Failure Recovery Rate Measures whether the agent can recover from recoverable tool failures. For example, it may retry a read-only request after a temporary timeout. Retries of consequential actions require additional safeguards against duplicate execution. Average Tool Calls per Successful Task Measures execution efficiency. Lower values are not always better. A workflow that skips necessary validation may use fewer calls while producing an unsafe outcome. End-to-End Latency Measures the time required to complete the full task. Track percentile latency rather than relying only on averages. Cost per Successful Task Measures the total cost of successfully completed tasks. Include model inference, external APIs, and relevant infrastructure expenses. How to Evaluate Agents With Multiple Valid Tool Paths Exact trajectory matching is useful when a workflow has strict ordering requirements. However, many tasks can be completed through several valid sequences. Consider a research agent that must gather information from two independent sources. The agent could retrieve Source A before Source B or retrieve Source B before Source A. Both approaches may be correct. An evaluator that requires one exact sequence would incorrectly reject one of them. Use Invariants Instead of Exact Sequences An invariant is a condition that must remain true regardless of the execution path. Examples include: - The agent must verify authorization before a protected action. - The agent must not execute prohibited tools. - The agent must use valid identifiers. - The agent must not perform the same consequential action twice. - The final state must satisfy the task requirements. Evaluate Partial Ordering Some actions must happen before others, while independent operations can occur in any order. For example: Customer Lookup | v Eligibility Check | v Approval | v Refund Execution The evaluator can require these dependencies without requiring every intermediate read-only operation to occur in one exact position. Verify State Transitions For agents that modify external systems, checking the final state is often more reliable than comparing generated messages. For example: Initial State: Refund Status = Not Started Expected Final State: Refund Status = Completed Approval = Verified Refund Count = 1 This approach measures whether the system reached the intended outcome. It should be combined with checks for prohibited intermediate actions, since a correct final state does not prove that every action was authorized. Common Mistakes in Agent Tool-Use Evaluation Evaluating Only the Final Response An agent may claim success without performing the required action. Inspect actual tool execution and final system state. Treating Tool Selection as Task Completion Calling the correct tool does not guarantee a successful outcome. The arguments, execution result, and final state also matter. Requiring One Exact Tool Sequence for Every Task Some workflows permit multiple valid paths. Use invariants and partial-order checks when appropriate. Ignoring Unauthorized Actions An agent may complete the requested task while performing prohibited operations. Safety and authorization must be evaluated independently. Running Tests Against Live Production Systems Write-capable agent tests can modify records, send messages, or trigger financial operations. Use isolated environments and test fixtures wherever possible. Relying Entirely on LLM-Based Judges LLM evaluators can make mistakes. Use deterministic checks for schemas, permissions, and state transitions when possible. Ignoring Tool Failure Scenarios Agents must handle timeouts, invalid responses, and unavailable services. Test these conditions explicitly. Measuring Efficiency Before Correctness An agent that uses fewer tool calls is not necessarily better. Correctness, authorization, and task completion should take priority. Frequently Asked Questions What is agent tool-use evaluation? Agent tool-use evaluation measures whether an AI agent selects appropriate tools, supplies correct arguments, follows required workflows, handles execution results, and completes tasks successfully. Which framework is best for evaluating agent tool calls? The answer depends on the evaluation requirements. DeepEval and Promptfoo are useful for automated tool-call checks. Inspect AI supports controlled execution environments. LangSmith and Braintrust provide trace-linked evaluation workflows. AgentBench is more suitable for standardized research benchmarking. What is the difference between tool correctness and task completion? Tool correctness examines whether the agent selected and called appropriate tools. Task completion measures whether the intended outcome was achieved. An agent can call the correct tools and still fail the task. How do you evaluate tool arguments? Validate the argument schema and compare values against expected inputs. For sensitive operations, also verify that arguments correspond to authorized resources. Should agent evaluations check execution order? Yes, when order affects correctness or safety. For example, authorization should precede protected actions. However, exact ordering is unnecessary for independent operations that can occur in any valid sequence. Can LLM-as-a-judge evaluate agent trajectories? Yes. A judge model can assess whether an execution path appears appropriate. However, its judgments should be calibrated against human-reviewed examples and supplemented with deterministic checks. How do you test agents that modify external systems? Use isolated test environments with controlled data. Record tool calls and verify resulting state changes. Avoid running destructive or consequential evaluations against live production systems. How do you measure agent efficiency? Track tool calls, model requests, latency, resource usage, and cost per successful task. Efficiency should be considered alongside correctness and safety. Is AgentBench suitable for enterprise agent testing? AgentBench is useful for general agent benchmarking. However, enterprise applications also require tests involving their actual tools, permissions, workflows, and operating conditions. How often should agent evaluations run? Important changes to models, prompts, tools, and orchestration should trigger relevant regression tests. Production teams should also review representative traces and add newly discovered failures to their evaluation datasets. Final Recommendations Evaluating AI agents requires examining both their decisions and their outcomes. Choose Inspect AI when controlled execution environments and reproducible agent evaluations are important. Consider DeepEval for Python-based tool correctness, argument validation, and task completion testing. Evaluate LangSmith when trajectory analysis and execution tracing are central requirements. Consider Braintrust for centralized datasets, experiments, and custom agent scoring. Use Promptfoo when deterministic tool-call assertions and CI regression testing are priorities. Evaluate AgentBench when comparing general agent capabilities across interactive environments. The best evaluation strategy may combine more than one approach. For example, a team could use Promptfoo for tool-call regression testing, Inspect AI for controlled execution tasks, and a tracing platform for investigating production behavior. The goal is not to adopt the largest number of frameworks. It is to verify that agents select appropriate tools, execute authorized actions, handle failures correctly, and achieve the intended outcomes without introducing unacceptable operational risk.

ControlArena: the monitored model could shape its monitor’s prompt In AI control experiments, a trusted monitor reads what an untrusted model did and scores how suspicious it looks. That score means something only if the untrusted model cannot change the structure of the prompt the monitor reads. In March 2026 I found that it could in ControlArena, the UK AI Security Institute's library for AI control research. Two monitor prompt formatters, including the default one, put the monitored model's tool-call arguments into the prompt as raw text, with nothing marking where that text ended. I reported it in ControlArena issue #791 and sent a patch in PR #795. The maintainer, @Rogan-Inglis, merged it with formatting fixes as PR #798. My follow-up for the chain-of-thought formatter, PR #810, was merged three weeks later. What the monitor read The last_tool_calls formatter, the default for basic_monitor, and the prefix formatter built each tool call with an f-string: tool_calls_message += f"{tool_call.function}: {tool_call.arguments}\n" The arguments went in as a Python dictionary repr, and the result was substituted into the monitor's prompt template. The monitored model chooses every character of those arguments. A model trying to get past its monitor could put text in an argument that reads like part of the prompt: a closing tag, a new heading or a note addressed to the monitor. Nothing in the prompt showed the monitor where the model's text stopped and the experimenter's instructions resumed. The library already had a safer pattern. Its XML trajectory formatter passed every model-derived field through html.escape(), so escaped content could not close its element. The two tool-call formatters did not. The accepted fixes #798 serializes tool-call arguments with json.dumps() instead of Python repr, and HTML-escapes them inside the prefix formatter's tags so an argument cannot forge a closing tag. It follows the convention the full-trajectory formatter already used. Its tests cover closing-tag injection, HTML in function names and fake monitor instructions. #810 closes the same gap in the chain-of-thought formatter, which put the monitored model's reasoning into the prompt with no boundary. The reasoning now sits inside <cot>...</cot> tags with its content HTML-escaped, and new tests cover closing-tag and fake-tag injection. Both changes are marked as breaking. The monitor now sees different text, so the maintainers note that experiment results from before and after each change are not directly comparable. That note is a finding in itself: how the evidence is laid out in a monitor's prompt is part of the measurement. The same failure in AI evaluation This is ETB-02 in the Evaluator Trust Boundary taxonomy: an evaluator's prompt treats text written by the evaluated model as part of its own structure. LLM judges have the same shape, since a judge reads text written by the system it grades. The ETB-02 page lists every public report in the class, including the same problem in the judge prompt of UK AISI's Inspect. Source record - ControlArena issue #791: report. - ControlArena PR #795: my patch. - ControlArena PR #798: accepted fix, by @Rogan-Inglis, based on #795. - ControlArena issue #809 and PR #810: the chain-of-thought follow-up. - ETB-02: unsanitized model output in the evaluator prompt. - My AI evaluation contribution record. This note describes the reports and the accepted changes. Escaping stops the monitored model from forging the prompt's structure. It does not make a monitor immune to persuasive text inside that structure, which is a question about the monitor model itself.

I shut down my agent-safety toolkit to audit AI graders Earlier this year I posted here about SafeClaw and Authensor's policy engine for AI agents. That product line is retired, and those posts now say so at the top. This is what Authensor works on instead. Authensor is an AI evaluation integrity firm in San Francisco. It audits the tools the AI industry measures itself with: evaluation harnesses, benchmarks, LLM judges and scoring pipelines. The question for each one is whether the system being evaluated can move its own score. Why the switch Two of the early bugs I fixed upstream were in agent-control tooling. In April 2026, UK AISI's ControlArena passed tool-call arguments and chain-of-thought text into its monitor prompts unsanitized, so the agent being monitored could write straight into the prompt of the model watching it. The same pattern kept turning up in graders, judges and benchmarks. If the measurement can be steered by the thing being measured, the safety numbers built on it don't mean much, so that became the work. The Evaluator Trust Boundary The research is organized around one definition: an evaluator has an Evaluator Trust Boundary (ETB) failure when its score depends on an artifact the evaluated system can influence. The taxonomy names ten mechanisms, ETB-01 to ETB-10. The preprint is on Zenodo: 10.5281/zenodo.21633620. Some of the cases now fixed upstream: - SWE-bench (August 2026, ETB-09): the grader counted PASSED lines that the patch under test could print itself. Fixed in PR #620. - Promptfoo (October 2026, ETB-03): an llm-rubric judge response with no verdict defaulted to a pass. Fixed in PR #10611. - UK AISI Inspect AI (June 2026, ETB-04): scoring metrics silently dropped errored and inconclusive samples from the denominator. Fixed in #4288. - NVIDIA garak (July 2026, ETB-06): string detectors did no Unicode normalization, so homoglyph and zero-width variants got through. Fixed in #1884. - Instructor (July 2026, ETB-01): JSON extraction took the first object in a code block, so planted JSON outranked the real output. Fixed in #2434. - UK AISI ControlArena (April 2026, ETB-02): tool-call arguments entered monitor prompts unsanitized. Fixed in #798. The full list, with every report and fix, is at authensor.com. Separately, the MLflow advisory for CVE-2026-64849 credits Authensor as an independent finder. Check your own judge etb-scan is a free, MIT-licensed scanner that measures ETB-01 (injected verdicts) against your own judge. It runs offline on the standard library, with no API keys. pip install etb-scan etbscan --judge mypkg:judge --trials 10 If you've published an eval number and want it checked against the instrument that produced it, Authensor does that free within 48 hours: john@authensor.com. The hardware side 15 Research Lab is my separate lab in San Francisco for embodied AI safety and experimental hardware. The same kind of bug shows up there. Yosys's formal exporters were silently dropping every assertion, so solvers reported PASS on designs that should fail. I reported it as #6108 and Zaneham fixed it in #6117. Write-up. Links - ETB definition, cases and citation: authensor.com - Every report and fix: authensor.com - etb-scan: AUTHENSOR/etb-scan - 15 Research Lab: 15researchlab.com - Me: johndanielkearney.com

Inspect AI in Production: A Hard-Nosed Review of Agent Evals, Logs, and Model-Upgrade Gates Why We Evaluated Inspect AI The model upgrade itself is rarely the difficult part. The difficult part is proving that the new model did not silently damage extraction accuracy, tool behavior, refusal handling, or operational cost. <div class="ev-card" data-lm-figure="evcard-inspect-ai-eval" data-lang="en"><div class="ev-cardtitle">Inspect AI Verdict</div><div class="ev-cardrows"></div><p class="ev-card__takeaway">Inspect AI provides the execution and artifact layer for item-level agent-eval gates, but raw log reproducibility, scorer contracts, sandbox boundaries, and cost accounting remain engineering responsibilities.</p></div> We have seen the same internal harness emerge repeatedly: a JSONL dataset, an asynchronous request loop, a pile of retry logic, a scorer script, and a spreadsheet comparing the old model with the new one. It works until an incident forces us to answer questions the harness was never designed to answer: - What exact prompt and tool trajectory produced this result? - Was the failure caused by the model, the grader, the provider, or the sandbox? - Which individual samples regressed? - Did the replacement model consume more tokens despite improving pass rate? - Can we reproduce the evaluation from a pinned task and immutable artifact? We evaluated Inspect AI because it handles that entire execution layer. An Inspect evaluation combines a dataset, a solver, and one or more scorers. The solver may be a single generation call, a custom function decorated with @solver, or a multi-turn agent. The scorer may perform exact matching, inclusion matching, model grading, or arbitrary custom logic. The framework also writes structured .eval logs and ships the inspect view browser interface. That separation matters. We do not want the dashboard to be the source of truth; we want the artifact to be the source of truth and the dashboard to be one way of reading it. The agent surface was another reason for testing it. Inspect accepts agents in the solver position and includes a ReAct loop, tool interfaces, message and token limits, custom agents, agent handoffs, and adapters for external agents. Its tool catalog includes shell, Python, text editing, web, computer, and MCP-oriented interfaces. For dangerous execution, it supports sandbox backends including Docker and more infrastructure-heavy options. We also inspected the Inspect Evals repository rather than evaluating the framework only with toy arithmetic. It provides a substantial task corpus and demonstrates how real evaluations organize datasets, task versions, dependencies, scoring, and sandbox assets. We still recommend pinning a small subset into an internal regression suite rather than importing a moving benchmark wholesale. Our short conclusion before the details: Inspect gives us a much stronger foundation than another home-grown request loop. It does not, by itself, make a benchmark deterministic, a model grader trustworthy, or a tool safe. Hands-On Walkthrough: Setup, Execution & Output We ran the offline scorer check with Python 3.12 in an isolated container. For the local walkthrough, we recommend a virtual environment. That was deliberate: Python 3.11 and 3.12 are the Inspect Evals project's preferred targets, while later Python versions carry compatibility caveats for parts of the broader task corpus. For the local walkthrough, use this installation path: python3.12 -m venv .venv source .venv/bin/activate python -m pip install --upgrade pip python -m pip install "inspect-ai==0.3.100" openai export OPENAI_API_KEY="replace-me" mkdir -p logs The pinned inspect-ai wheel resolved successfully in our environment. The wheel we downloaded had this SHA-256 digest: 78e4c5b7e426fcc25563cd68f9f976a1c18c00adc9f4955bde9a6c97016363d7 For production, we would pin the complete lockfile rather than only the top-level package. Inspect pulls in a nontrivial dependency tree that includes asynchronous HTTP, schema, filesystem, terminal UI, and cloud-storage packages. We then reduced the task shape to the smallest useful production example: registered task, registered solver, custom partial-credit scorer, stable sample IDs, and an explicit log directory. # regression_eval.py import os import re from inspect_ai import Task, eval, task from inspect_ai.dataset import Sample from inspect_ai.scorer import Score, Target, scorer from inspect_ai.solver import Generate, TaskState, solver @solver def answer_once(): async def solve(state: TaskState, generate: Generate) -> TaskState: return await generate(state) return solve def normalize(text: str) -> str: return re.sub(r"\s+", " ", text.strip().lower()) @scorer def partial_answer(): async def score(state: TaskState, target: Target) -> Score: answer = normalize(state.output.completion) expected = normalize(target.text) if answer == expected: value = 1.0 reason = "Exact normalized match" elif expected in answer: value = 0.5 reason = "Target present with additional text" else: value = 0.0 reason = "Target absent" return Score( value=value, answer=state.output.completion, explanation=reason, ) return score @task def upgrade_gate(): return Task( dataset=[ Sample( id="refund-window", input="Reply with only the refund window: 30 days", target="30 days", ), Sample( id="support-tier", input="Reply with only the support tier: enterprise", target="enterprise", ), ], solver=answer_once(), scorer=partial_answer(), ) if __name__ == "__main__": eval( upgrade_gate(), model=os.environ.get("INSPECT_MODEL", "openai/gpt-5"), log_dir="logs", ) To execute this worked example through Python or the CLI and inspect its logs, use: export INSPECT_MODEL="openai/gpt-5" python regression_eval.py # Equivalent task discovery through the CLI inspect eval regression_eval.py@upgrade_gate \ --model "$INSPECT_MODEL" \ --log-dir logs inspect view --log-dir logs A successful run should produce output in this general shape. The following console block is illustrative, not a measurement from our failed qualification run: $ inspect eval regression_eval.py@upgrade_gate \ --model openai/gpt-5 \ --log-dir logs upgrade_gate (2 samples): complete Model: openai/gpt-5 Log: logs/2026-10-11T101530_upgrade-gate.eval Scores: partial_answer: 0.750 Samples: completed: 2 errors: 0 For an agent task, the task definition changes more than the execution command. This is the minimal pattern we would use before adding a pinned challenge dataset: from inspect_ai import Task, task from inspect_ai.agent import react from inspect_ai.scorer import includes from inspect_ai.tool import bash, python, todo_write @task def sandbox_agent_eval(): return Task( dataset=load_pinned_samples(), solver=react( prompt="Use the available tools and submit only the final answer.", tools=[bash(), python(), todo_write()], attempts=3, ), scorer=includes(), sandbox="docker", message_limit=30, ) The important production property is not the decorator syntax. It is that sample inputs, targets, messages, tool calls, outputs, scores, and run configuration can travel together in one reviewable artifact. Evaluation Failures and Verification Limits Our first narrow scorer contract test did not complete successfully. We created five fixed completion fixtures to compare includes() with match(location="exact"). One fixture was intentionally labelled as an incomplete-flag negative case. Our test expected the exact matcher to return incorrect, but Inspect returned C, causing this assertion failure: AssertionError: ('incomplete_flag_negative', 'exact', 'C', 'I') We do not treat that as proof that Inspect's scorer is defective. It shows that this fixture's expected result did not match the scorer's returned value. The assertion alone does not establish whether normalization, answer extraction, fixture construction, or another factor caused the mismatch. Because the assertion terminated the run, we did not obtain a complete five-fixture result set. That failure changed our implementation policy: every built-in scorer we adopt gets a table-driven contract suite containing punctuation, surrounding prose, denial text, case changes, whitespace, multiple answers, and truncated targets. A scorer name is not a specification. Partial credit introduces another interpretation problem. A score set of 0, 0.5, and 1 is easy to emit, but the business meaning of its aggregate is ours to define. A plain arithmetic mean treats two partial passes as equivalent to one full pass and one complete failure: mean([0.5, 0.5]) == mean([1.0, 0.0]) == 0.5 Those distributions carry different operational risk. We therefore preserve at least four outputs: full-pass rate, partial-pass rate, hard-failure rate, and mean score. We would not approve a model upgrade from the mean alone. We have not established byte-for-byte .eval reproducibility. Our offline scorer check did not run evaluations or compare log files; log reproducibility requires a separate experiment. Consequently, we have no defensible raw SHA-256 comparison to publish. Even after a successful run, we would separate two requirements: 1. Artifact integrity: the stored .eval file remains unchanged after creation. 2. Semantic reproducibility: equivalent runs contain the same sample outputs and scores after excluding run-specific metadata. Raw byte equality is a stricter test. Run IDs, timestamps, ordering, provider metadata, and archive serialization can invalidate it even when the evaluation result is semantically identical. Our CI gate would hash the original artifact for custody, then generate a normalized comparison document for regression analysis. Model grading creates another source of nondeterminism. Exact match is brittle but inspectable. A model grader is flexible but adds a second model call, another prompt, another model version, and another failure mode. We did not complete a valid exact-match-versus-model-grader drift experiment, so we will not invent a disagreement rate. Our deployment rule is still clear: we pin the grader independently, store its explanation, and maintain a human-adjudicated calibration set. The sandbox boundary also needs precise language. Adding bash() does not automatically isolate anything. In our task definition, sandbox="docker" is what assigns shell and Python execution to the container. We would not assume that custom Python tools are isolated; we would verify their execution context and route dangerous operations through a sandbox-aware interface. We therefore treat the evaluator host as sensitive infrastructure: - We run it with minimal credentials. - We mount only required fixtures. - We remove Docker socket access from evaluated containers. - We disable outbound networking unless the task requires it. - We apply message, token, time, and cost limits. - We assume model-generated shell commands are hostile. Finally, we did not complete a provider rate-limit benchmark or a two-model token-cost comparison. The available run failed before remote inference. We cannot publish retry counts, wall time, throughput, or cost per sample from that attempt. The workaround is procedural, not rhetorical: record provider errors as their own failure category, pin concurrency, preserve token usage per sample, and rerun the exact same sample IDs against both model versions. We would reject any benchmark table that silently drops exhausted retries. Scale, Latency & Cost vs. Alternatives Inspect's strongest comparison is not against an observability dashboard. It is against the bespoke evaluation runner that engineering teams eventually build around raw provider SDKs. | Option | Execution model | Item-level scoring | Agent trajectories | Isolation | Regression artifacts | Main cost | |---|---|---:|---:|---:|---:|---| | Inspect AI | Local Python runner with provider APIs | Strong | Strong | Explicit sandbox configuration | Structured .eval logs | Integration and evaluation design | | Bespoke Python harness | Whatever we implement | Variable | Usually custom work | Usually custom work | Usually JSONL or database rows | Engineering ownership | | Langfuse or Phoenix-style observability | Trace collection and analysis | Possible, but not the core runner | Strong tracing | Outside the primary scope | Trace-oriented | Backend operation and instrumentation | | Prompt-oriented CI harness | Configuration-driven test execution | Strong for prompt and security checks | Depends on integration | Depends on provider and tool setup | CI reports and artifacts | Configuration growth | | Commercial evaluation platform | Hosted execution and dashboards | Usually strong | Product-dependent | Product-dependent | Vendor-managed | Subscription, data governance, lock-in | We did not produce valid latency or throughput numbers, so we do not present fictional requests-per-second figures. For our deployment benchmark, we would measure provider latency, agent trajectory length, tool execution time, and local orchestration overhead separately before identifying the dominant bottleneck. Our cost gate uses artifact-derived quantities: sample_cost = input_tokens × input_price_per_token + output_tokens × output_price_per_token + grader_input_tokens × grader_input_price_per_token + grader_output_tokens × grader_output_price_per_token + external_tool_cost We calculate the model-upgrade delta over identical sample IDs: full_run_delta = sum(new_model_sample_costs) - sum(old_model_sample_costs) We then combine quality and cost: cost_per_additional_full_pass = full_run_delta / (new_full_pass_count - old_full_pass_count) If the denominator is zero, this ratio is undefined; an upgrade can still improve cost efficiency by preserving quality at a lower full-run cost. Equal full-pass counts alone do not establish preserved quality, so we also review item-level regressions and failure categories. If the denominator is negative, we assess the quality loss and cost change separately rather than interpreting the ratio as a cost per additional full pass. If model grading is enabled, grader tokens stay separate from candidate-model tokens so that a grading prompt change cannot masquerade as candidate-model cost growth. The build-versus-buy calculation is similarly direct: annual_internal_harness_cost = initial_engineering_hours × loaded_hourly_rate + annual_maintenance_hours × loaded_hourly_rate + incident_and_audit_overhead annual_inspect_cost = integration_hours × loaded_hourly_rate + task_maintenance_hours × loaded_hourly_rate + model_and_tool_usage + sandbox_infrastructure Inspect is open source, but it is not free to operate. The expensive parts are representative datasets, reliable scorers, model calls, sandbox execution, and review of ambiguous failures. Inspect removes a large amount of harness plumbing; it does not remove evaluation engineering. For broader implementation help, we keep our infrastructure patterns in the [Effloow tools collection](/tools). For teams turning an ad hoc benchmark into a release gate, our [AI engineering services](/services) cover dataset versioning, grader calibration, and CI integration. Our Final Verdict: When to Deploy, When to Skip We would deploy Inspect AI as the execution and artifact layer for a model-upgrade gate, with conditions. Deploy it if: - We need repeatable item-level comparisons rather than aggregate dashboard metrics. - We evaluate multi-turn agents, tool calls, or sandboxed code execution. - We want task, solver, and scorer logic stored as reviewable Python. - We need transcripts and scoring explanations for incident review. - We are willing to pin framework, task corpus, dataset, model, grader, and pricing versions. - We can maintain scorer contract tests and a human-adjudicated calibration subset. - We want to avoid building logging, resumption, trajectory capture, and visualization from scratch. Hold off or avoid it if: - We only need a few deterministic HTTP assertions in CI. - We require raw byte-identical logs and have not built normalization and integrity checks. - We expect model-graded scores to be stable without pinning and calibration. - We cannot isolate model-generated code from evaluator credentials and host resources. - We need a fully managed dashboard with organizational workflows more than a Python evaluation framework. - We are unwilling to inspect failed samples and only want a single pass-rate number. Our production verdict is adopt with explicit validation and operational controls. The component model is clean, the agent support is substantive, the task corpus is useful, and inspect view makes trajectory debugging materially easier. Inspect can replace weeks of basic harness construction. Our own validation was limited, however. Installation succeeded, but the offline scorer check terminated on an assertion mismatch before producing a complete five-fixture result set. Log reproducibility, provider rate-limit behavior, model-grading drift, and per-sample upgrade cost were outside that check's scope and remain untested. We would not promote that check into a production benchmark table. That restraint is part of the recommendation. Inspect provides the execution and logging components for an auditable release gate. We still have to prove the scorer, define the sandbox boundary, preserve the artifacts, and make the economics explicit. If those controls are already on your roadmap, Inspect is one of the first frameworks we would prototype. If you want help designing the gate around your own failure modes, [contact Effloow](/contact).