Termination, Loop Detection, and Safe Stopping

Step budget, Progress rule, and Model self-termination stop a run in three ways. Only one of them reads the task state. The success criterion is the same for all three: a successful terminal state includes independently verifiable evidence. Something a reader outside the run can check, rather than the agent's own assertion that the work is over.

The step budget is the oldest of the three, and it did not come out of agent frameworks. A provable loop bound has been Rule 2 of ten rules for safety-critical code since 2006. Gerard J. Holzmann wrote those rules at the NASA/JPL Laboratory for Reliable Software. Rule 2 reads: "All loops must have a fixed upper-bound. It must be trivially possible for a checking tool to prove statically that a preset upper-bound on the number of iterations of a loop cannot be exceeded. If the loop-bound cannot be proven statically, the rule is considered violated." The rationale is one clause: "The absence of recursion and the presence of loop bounds prevents runaway code." The last sentence of the rule is the part worth copying. A bound nobody can prove before the program runs is not a weaker safeguard. It is a violation.

That is the honest description of the first column. A step budget is easy to enforce, because a counter needs to know nothing about the task. For the same reason it will cut off useful work, and will happily let useless work run to the ceiling. Model self-termination is the mirror image. Maximally flexible, and unverifiable by construction, because the only witness to completion is the party being tested. The Progress rule sits between them. It is the only one of the three that reads a task-specific state measure. That is what lets it separate legitimate polling from repeated non-progress. The price is a metric someone actually defined.

The failure sits at the other end of all three, and it wears two faces. Premature success and endless continuation share one root cause. Diagnose them together, rather than patching them one at a time.

τ-bench is a customer-service agent benchmark. Its interest here is not the score but the scorer. The τ-retail domain holds 115 tasks over a database of 500 users, 50 products and 1,000 orders. A run earns a rule-based reward, r = r_action × r_output. r_action checks that the final database is identical to the unique ground-truth outcome database. r_output checks that the agent's replies contain the required information as substrings. The 2024 paper puts it in one line: "Our evaluation scheme compares the database state at the end of each episode with the ground truth expected state." Nothing in that reward reads the agent's account of what it did.

The numbers show what that outside check catches. On a 115-task τ-retail sample with one trial per task, the gpt-4o function-calling agent solved 75 and failed 40 — pass^1 = 65.2%. Its pass^8 is the fraction of tasks it can get right eight times running. In τ-retail that dropped below 25%. In the main results table the same agent scores pass^1 = 61.2% on τ-retail and 35.2% on τ-airline. Two thirds of the time the run reached the right end state. Asked to reach it repeatedly, it reached it on fewer than a quarter of the tasks. A stopping rule that trusted the agent's own conclusion would have recorded neither gap.

Agent termination should be based on task state, verified completion, irrecoverable failure, user cancellation, policy refusal, escalation, or exhausted budget. "The model says it is done" is not sufficient.

These failures are not hypotheses. They are counted categories. MAST, the Multi-Agent System Failure Taxonomy, was built at UC Berkeley in 2025 from more than 1,600 annotated traces across 7 multi-agent frameworks. Its 14 failure modes were derived from 150 of those traces, at an inter-annotator agreement of kappa = 0.88. One of the 14 is defined in the appendix as "FM-1.5: Unaware of termination conditions - Lack of recognition or understanding of the criteria that should trigger the termination of the agents’ interaction, potentially leading to unnecessary continuation."

The taxonomy's failure breakdown puts numbers on both halves of the loop problem. Step repetition (FM-1.3) accounts for 15.7%, and not recognizing task completion (FM-1.5) for 12.4%. Premature termination (FM-3.1) accounts for 6.20%, no or incomplete verification (FM-3.2) for 8.20%, and incorrect verification (FM-3.3) for 9.10%. Stopping too early, never stopping, and repeating a step are not edge cases at the margin of agent design. Between them these five modes carry a large share of the observed failures. Three of the five are about verification rather than about capability.

Loop detection compares actions, states, progress, and unresolved blockers across iterations. It should distinguish legitimate polling from repeated non-progress. That is the distinction FM-1.3 makes countable, and the one no single-step view of a trace can make at all.

A run can end in four ways worth naming separately. Success terminal: the contract's completion evidence is verified. Partial terminal: useful artifacts exist, but the full outcome cannot be completed. Escalation terminal: a human or another service must decide the next step. Failure terminal: a non-recoverable error or policy block prevents continuation. A fifth, Cancellation terminal, is the principal withdrawing the task or the authority. It and Failure terminal should sit with different owners, and under different tests.

The clearest instance of the last two is older than the field. NASA's Remote Agent was the first autonomous control architecture to run as flight software on an active spacecraft. On 18 May 1999 it deadlocked in flight on Deep Space 1. A missing critical section left two threads each waiting for an event only the other could supply. Thrusting did not turn off as requested. The spacecraft could not recover by itself. The run did not detect its own condition, and it had no terminal state to enter. So the terminal state came from outside it: "The Remote Agent experiment was immediately terminated from ground, and the space craft put in stand-by mode."

The accounting afterwards is what makes this a Partial terminal rather than a Failure terminal. The anomaly appeared after roughly 70% of the experiment's objectives had been met. The remaining 30% were run successfully the following Friday. A run that stops on a blocker with 70% of its objectives banked and a named next actor is a different outcome from a run that stops with nothing. A terminal design that cannot tell those two apart will report both as failure — or, worse, both as success.

Specify terminal logic for one agent you actually run, against a real workflow rather than a diagram. The logic is finished when you can point to two moments. One where it would cut off a run that has begun repeating itself. One where it would refuse a finish that nothing outside the agent can confirm. Both have to come from the workflow you picked, not from a sketch of one.

Five pieces, in order. Define success evidence: name the exact postconditions required for completion — τ-bench's answer, a comparison against the ground-truth end state, is the model to copy. List blockers: missing permission, unavailable tool, policy conflict, unresolved ambiguity, each with the actor who resolves it. Create loop signatures: track repeated action-state pairs, unchanged artifacts, and recurring errors. Set layered budgets: limit steps, time, cost, retries, and external effects. Design terminal responses: return evidence, partial artifacts, blockers, and the responsible next actor.

The middle two steps already have regulatory wording available, written for automated order routers rather than for agents. Broker-dealers must have controls reasonably designed to "Prevent the entry of erroneous orders, by rejecting orders that exceed appropriate price or size parameters, on an order-by-order basis or over a short period of time, or that indicate duplicative orders". That requirement is 17 CFR 240.15c3-5(c)(1)(ii). It is a loop signature and a layered budget in one sentence: a per-action check, a rate check over a short window, and an explicit test for duplication.

The price of omitting the halt is on the record. On 1 August 2012 Knight Capital Americas LLC's SMARS router turned 212 parent orders into over 4 million executions in 154 stocks, for more than 397 million shares in about 45 minutes. The loss was over $460 million, and the SEC's order of 16 October 2013 imposed a $12 million penalty. That order names the gap directly: "Knight also did not have procedures in place to halt SMARS’s operations in response to its own aberrant activity." 212 in, 4 million out, 45 minutes. The ratio between the input and the output is the loop signature, and no component inside the run was responsible for reading it.

Termination is a first-class design problem. Define it before building planning or memory, because every other mechanism otherwise creates more ways to continue incorrectly. The evidence for that ordering is in every case above. The runtimes that shipped counters shipped them because the model's judgment was not enough. The taxonomy that annotated more than 1,600 traces found step repetition at 15.7% and unawareness of termination conditions at 12.4%. The system that could edit its own code edited its own time limit. The router that turned 212 parent orders into over 4 million executions had no procedure for halting itself.

Before shipping, put the design through both questions again. What would make this agent stop too early, and what would make it never stop at all? A GPT-4 agent given the option to declare a task impossible took it on 54.9% of feasible tasks, so the first question is not the theoretical one. Then hold the stop condition to the same bar as everything else. The agent reporting that the work is done is not evidence that it is. Holzmann's formulation is the standard to borrow. A bound you cannot prove before the run counts as violated, not as nearly good enough.