Agentic Systems as Controlled Action

A retailer connected a language model to its refund API. The prototype answered questions well. Then it issued duplicate refunds when a timeout hid the first successful transaction.

That fault is not an unlucky detail of one prototype. It is the opening scenario of the IETF's 2025 draft on the Idempotency-Key header, which begins: “Consider a scenario where the client sent a POST request to the server, but the request timed out. The client is left uncertain about the status of the resource. It doesn't know if the resource was created or updated, or if the server even completed processing the request.”

The uncertainty is the whole problem. The caller cannot tell a lost request from a lost reply. It has exactly two options, retry or don't, and they are a double payment and a missing one.

The payments industry standardised the fix at the action interface. Stripe's API accepts a client-generated idempotency key on all POST requests. It saves the status code and body of the first request made for a given key. A later request carrying the same key returns that saved result instead of re-executing the payment. Keys can be removed once they are at least 24 hours old.

The control lives in the tool contract, not in the model's wording. No amount of prompt engineering makes a POST idempotent.

An agentic system combines a model with state, tools, control logic, and an environment. Its defining property is not conversational style. It is that the system can choose and execute actions over multiple steps.

A model output is a proposal. The surrounding runtime decides what is permitted, records what happened, and determines whether the task should continue.

What a runtime that permits every proposal costs is on the public record. On 1 August 2012 Knight Capital's automated equity order router, SMARS, went to work on 212 small retail orders. The SEC's order describes what followed: “While processing 212 small retail orders that Knight had received from its customers, SMARS routed millions of orders into the market over a 45-minute period, and obtained over 4 million executions in 154 stocks for more than 397 million shares.” Knight lost over $460 million.

On 16 October 2013 the SEC censured Knight Capital Americas LLC and imposed a $12 million civil money penalty. Among the failures it named: no controls reasonably designed to prevent the entry of erroneous orders at a point immediately prior to the submission of orders to the market.

212 orders in, more than 397 million shares out, in 45 minutes. Nothing in that sequence was deceptive. The proposer proposed and the runtime executed. That is exactly what a runtime with no boundary does.

Four things have to be written down before an agent can be called controlled: the goal and contract, the observation and state, the action interface, and the feedback and control loop. Feedback and control belongs to one owner, accountability to another. Each needs its own tests.

The action interface is not a blank sheet. US securities law already puts the boundary in the runtime rather than in the trading logic. SEC Rule 15c3-5 requires controls that “Prevent the entry of erroneous orders, by rejecting orders that exceed appropriate price or size parameters, on an order-by-order basis or over a short period of time, or that indicate duplicative orders.” Its title says where those controls sit: “Risk management controls for brokers or dealers with market access”.

Read that clause against the refund case. The duplicate is to be rejected by the interface, on an order-by-order basis or over a short period of time, before it ever reaches the market. Not detected in a reconciliation the next morning. Not excused by a well-worded explanation.

It is the refund assistant's failure named in law, written a decade before anyone connected a language model to a payments API. The SEC's order against Knight applied exactly that paragraph.

Autonomy here is measured by how much of the loop runs without anyone being asked again. Direct response sits at one end and Open-ended agent at the other, with Agentic workflow between them. A design in this space is working when the final environment state matches the declared task goal. It is failing when fluency conceals an unsafe action boundary: a smooth trace and a dangerous one read the same.

The cost of moving right along that scale has been measured. OSWorld, presented at NeurIPS in 2024, built 369 real computer tasks. Each one ships a fixed initial-state setup and a custom execution-based evaluation script. That is what containment costs in practice. For open-ended work there is no generic grader, so the researchers wrote one bespoke checking script per task, 369 times over.

Then the scores: “While humans can accomplish over 72.36% of the tasks, the best model achieves only 12.24% success, primarily struggling with GUI grounding and operational knowledge.”

“Harder to evaluate and contain” is the polite version of those two figures. Choosing the Open-ended agent column means accepting the distance between 72.36% and 12.24%. It also means paying for the per-task machinery that reveals it.

Pick one workflow that already runs in production and write its first agent contract. The exercise has worked when the contract shows you where a confident-sounding step could cross a boundary it should not cross. It has worked, too, when it leaves behind something a reviewer can check afterwards, to confirm the environment ended in the state the task asked for.

Two of the five steps can cite a statute rather than a preference. High-risk AI systems must “technically allow for the automatic recording of events (logs) over the lifetime of the system”. That is Article 12(1) of the EU Artificial Intelligence Act, of 13 June 2024. It fixes what “Define the evidence” must leave behind, and makes it the runtime's obligation rather than a reviewer's request.

A stop is required as well, and required to be a particular kind of stop. Article 14(4) says the system must reach the deployer in a form that enables the natural persons assigned to human oversight, as appropriate and proportionate, “to intervene in the operation of the high-risk AI system or interrupt the system through a ‘stop’ button or a similar procedure that allows the system to come to a halt in a safe state.”

That is “Design a fallback” with its acceptance criterion supplied: not merely a stop, but a halt in a safe state. And it presumes “Assign authority” has already been done. The article is written about named natural persons who must be able to intervene. A contract that cannot say who they are does not yet meet a standard published in 2024.

Use the word “agent” only when the system can observe consequences and choose another action. A one-shot model call with no environment loop is better described as a model-powered feature. Before approving a run, a reviewer asks two questions. Is the smoothness of the trace hiding an action the agent should not have been allowed to take? Is the world it left behind the one the task actually described?

The second question has a measured answer. WebArena, presented at ICLR in 2024, releases tasks “focusing on evaluating the functional correctness of task completions” rather than the text an agent produces. It reports that “our best GPT-4-based agent only achieves an end-to-end task success rate of 14.41%, significantly lower than the human performance of 78.24%.”

14.41% against 78.24% is a gap of 63.83 points. It appears only once completion is scored on the world instead of on the sentence. That is the entire argument for the word. A system worth calling an agent is one that can observe that it landed in the wrong state and choose again. A system worth approving is one whose record lets a reviewer see which state it landed in.