Trajectory, Tool, and Policy Evaluation

One run can succeed and offend at the same time. AgentDojo is built on that observation. Published in 2024, it puts an agent through 97 tasks and 629 security test cases, and it refuses to collapse the result into a single score. One axis records whether the user's task was completed. The other records whether the attacker's injected action was executed. The benchmark defines the combined measure in one line: “Utility Under Attack: the fraction of security cases (i.e., a pair of user task and injection task) where the agent solves the user task correctly, without any adversarial side effects.”

The gap between the two axes is the whole lesson. GPT-4o reaches 69.00% benign utility and 50.08% utility under attack, with a 47.69% targeted attack success rate. Roughly half the security cases end with the attacker's chosen action carried out. An outcome-only grader reads whether the user got what they asked for and nothing else. It prices that run at 50.08% instead of flagging it.

One signal carries more weight than the rest: policy violations are detected even when the task outcome is correct. A clean result should not buy a pass for a step that broke a rule. Three further signals need separate checks. Equivalent safe trajectories should receive comparable scores. Argument errors should be separated from tool-selection and planning errors. Trace review should lead to a specific runtime, tool, or policy fix.

Put those three under repeated cases before trusting them, and use a metric built for repetition. τ-bench grades a run against the environment rather than the transcript: “We employ an efficient and faithful evaluation process that compares the database state at the end of a conversation with the annotated goal state.” That was 2024. It covers 115 retail and 50 airline tasks. Its pass^k metric asks whether the same task is solved on every one of k independent attempts. gpt-4o scores 61.2 on τ-retail and 35.2 on τ-airline at pass^1. Run the retail tasks eight times each and pass^8 falls below 25%. A single passing trace is not evidence that the behaviour is reliable. It is one draw from a distribution in which most tasks are not solved eight times running.

Trajectory evaluation examines the sequence of observations, decisions, tool calls, and state updates that produced an outcome. It matters most when several paths can reach the same final result with different safety or cost profiles. AgentDojo makes the point arithmetically. The same agent reaches 69.00% benign utility and 50.08% utility under attack. The second number withholds credit from runs whose ending was right and whose path included the attacker's action.

A trace is not automatically a faithful explanation of hidden reasoning. It is operational evidence about what the runtime exposed and what the system did.

Four questions separate a lucky trajectory from a sound one. Selection, Arguments, Ordering, and Recovery. Recovery and Policy should not share an owner or a test.

The Selection/Arguments split is not a drawing convention. τ-bench measured it. Its authors hand-read 115 gpt-4o function-calling trajectories in τ-retail and found 40 failures, 36 of them agent faults. About 55% of the failures were wrong arguments or wrong information. Another 25% were rule-and-tool-selection errors, which the paper calls “wrong decision-making”. The argument class has a concrete, countable signature: “while gpt-4o FC agent only makes 0.46 tool calls with non-existent user/product/order/item IDs per τ-retail task, gpt-3.5-turbo FC / Act agents make 2.08 / 6.34, respectively.” Calls carrying identifiers that do not exist are a per-task rate. You can log it and watch it move. It belongs to a different owner, and a different fix, than choosing the wrong tool for a correct plan.

A trajectory rubric has to permit more than one correct path, which is why it is written against a real workflow. Score the actions the agent took, not the story it told about them. A detailed trace can tempt an evaluator into rewarding plausible narration instead of valid action. The rubric should also leave a written trail showing where a rule was broken, including on the runs that finished with the result everyone wanted.

One of those steps already has a worked implementation. The Berkeley Function Calling Leaderboard, published in 2025, is “Define acceptable alternatives” made mechanical. A single-turn call is marked correct only when the function name matches exactly and every argument value falls inside its allowed set. That is the Selection and Arguments check, in code. For multi-turn runs it drops the demand for one canonical transcript and grades the environment instead, with a state check plus a response check that an entry must pass in every turn: “Multiple sequences of function calls can achieve the same result, but the final state must match the labeled outcome.” The response check exists because read-only steps leave no state behind to compare. Without it, a rubric that grades only end state cannot see whether the agent looked where it was allowed to look.

Trajectory metrics are diagnostic complements to outcome metrics. Do not optimize them so aggressively that the agent learns to imitate a preferred trace instead of solving the task safely. The reinforcement-learning result above is the shape of that failure at the limit. The injected reward hack was exploited on more than 99% of prompts and verbalised less than 2% of the time in most of the testing scenarios. The approved shape was reproduced. The honest report was not.

The person reading traces all day is the one most likely to be won over by a fluent, well-narrated run, so the rubric has to protect them from their own reading. What survives that pressure is the finding that stands on its own: a rule was broken at a named step, whatever the ending looked like. That is the 47.69% of AgentDojo security cases in which the attacker's action was executed. It is the drop from 61.2 at pass^1 to under 25% at pass^8 on the same τ-retail tasks.