Workflow, Assistant, or Agent?

A deterministic workflow follows known transitions. An assistant recommends or drafts while a person controls execution. An agent decides which action to take next when the path cannot be specified completely in advance.

The boundary is not about how capable the model is. It is about who holds the sequence. Anthropic's engineering guidance drew it in two sentences in 2024: “Workflows are systems where LLMs and tools are orchestrated through predefined code paths. Agents, on the other hand, are systems where LLMs dynamically direct their own processes and tool usage, maintaining control over how they accomplish tasks.”

Holding the sequence is worth something measurable. Orchestration vendors sell it by name. AWS states the guarantee plainly: “Standard workflows have exactly-once workflow execution and can run for up to one year. This means that each step in a Standard workflow will execute exactly once.” Exactly once, for up to a year, is a property you can write into a control document. A model that decides its own next action offers no such property. The property is the absence of a decision.

Agentic flexibility is valuable only when it addresses genuine uncertainty. It should not replace stable business rules, database constraints, or ordinary orchestration. All of those already come with the exactly-once guarantee that autonomy trades away.

On 1 August 2012 an automated order router called SMARS was handed 212 incoming retail parent orders. It belonged to Knight Capital Americas LLC. The SEC's order records what the system did with them: “While processing 212 small retail orders that Knight had received from its customers, SMARS routed millions of orders into the market over a 45-minute period, and obtained over 4 million executions in 154 stocks for more than 397 million shares.”

The cause was not a model and not a hard problem. One of eight servers had missed the new deployment. That single skipped step turned a bounded routing task into a repeated-action loop with no counter on it. The loop ran for roughly 45 minutes before it was stopped. The loss was over $460 million. In 2013 the SEC censured Knight Capital Americas LLC and ordered a $12,000,000 civil penalty, its first enforcement action under the market access rule, Rule 15c3-5.

SMARS was not an autonomous agent. That is exactly why it is the right case to start from. The failure mode that costs $460 million is not clever misjudgement. It is an unbounded repetition of a step nobody was counting. Autonomy does not introduce that failure mode. It removes the fixed path that used to make the count possible.

An architecture choice is a sorting job before it is anything else. Four bands: stable procedure, bounded judgment, adaptive action, human decision. Whatever the other three end up holding, the human decision keeps its own owner and its own tests.

For high-risk systems the fourth band is not a matter of taste. The EU AI Act makes human oversight a design obligation rather than a review habit. Article 14(4)(d) requires that the overseer be able to disregard, override or reverse the system's output. Article 14(5) goes further and sets a headcount for the biometric identification systems listed in Annex III point 1(a): “For high-risk AI systems referred to in point 1(a) of Annex III, the measures referred to in paragraph 3 of this Article shall be such as to ensure that, in addition, no action or decision is taken by the deployer on the basis of the identification resulting from the system unless that identification has been separately verified and confirmed by at least two natural persons with the necessary competence, training and authority.”

Read that as an architecture constraint and it stops being abstract. Two natural persons, competent, trained and authorised. That is a control point with a number attached. It survives a design review because it was not negotiated inside the design review.

The first three bands are yours to allocate. Encode the valid sequence directly where it is known. Use a model for classification, extraction, or proposal inside the procedure, where judgment is bounded. Use an agent only where observations determine an unknown next step. The fourth band is where consequences, ambiguity, or legitimacy require review — and sometimes, as in Article 14(5), where the law has already counted the reviewers for you.

Measure first whether a simpler baseline reaches the required task success rate. That single number settles most architecture reviews on its own, and there is now a published version of it. WebArena, released in 2023, is a reproducible environment of 812 long-horizon web tasks across four self-hosted site types, drawn from 170 templates. Its own result: “The results demonstrate that solving complex tasks is challenging: our best GPT-4-based agent only achieves an end-to-end task success rate of 14.41%, significantly lower than the human performance of 78.24%.” Same 812 tasks, 14.41% against 78.24%. That gap is what the required success rate costs to reach on the autonomous path.

The cost side has units too. TheAgentCompany runs 175 tasks inside a self-hosted simulated software company, across twelve model backbones: “We find in experiments that the best performing model, Gemini 2.5 Pro was able to autonomously perform 30.3% of the provided tests to completion, and achieve a score of 39.3% on our metric that provides extra credit for partially completed tasks.” Gemini 2.5 Pro spent an average of 27.2 steps and $4.20 per task to get there. Claude 3.5 Sonnet reached 24.0%, or 34.4% with partial credit, at 29.2 steps and $6.30 per task. Proportionality is now arithmetic: dollars and steps per attempt, against a completion rate below one in three.

The last two criteria only count once they hold under repeated or adversarial cases. τ-bench built that requirement into a metric. Its pass^k is the chance that all k independent, identically distributed trials of the same task succeed, averaged across tasks. The abstract reports: “Our experiments show that even state-of-the-art function calling agents (like gpt-4o) succeed on <50% of the tasks, and are quite inconsistent (pass^8 <25% in retail).” Behind that sit per-domain single-run figures of roughly 61% on τ-retail and roughly 35% on τ-airline. In retail, then, the same system succeeds above 60% of the time once and under 25% of the time eight times running. One clean demonstration is a pass^1.

A review that tries to make the architecture smaller works best on a workflow someone has already shipped. Keep going until you can see where autonomy is buying you less than a clearer product decision would. Write down the numbers that show whether the plainer version already meets the target success rate.

Draw the current process first. Mark fixed transitions, judgment points, and human approvals. Note which of the approvals are the kind Article 14(4)(d) describes, where the overseer must be able to disregard, override or reverse the output.

Then classify the uncertainty, separating missing requirements from information that truly arrives at runtime. The 3 categories behind the 14 failure modes Cemri and colleagues counted are all in the first group, not the second.

Prototype the simplest option next: rules, one-shot model calls, and bounded workflows before an agent loop. Record what each scores. WebArena's 14.41% against 78.24% is only known because someone ran both.

Compare failure costs with real ones where they exist. The step asks for the effect of skipped steps, duplicate actions, and delayed clarification, and Knight Capital prices the first two together. A deployment step skipped on one of eight servers. Over 4 million executions in 154 stocks for more than 397 million shares, in roughly 45 minutes. Over $460 million lost, and a $12,000,000 civil penalty under Rule 15c3-5. Estimate against numbers of that kind, not against intuition.

Finally, document the choice. Record why additional autonomy earns its complexity, in the units the previous steps produced: success rate, steps and dollars per task, consistency across repeats, and the cost of the failure mode you are accepting.

A good architecture review can conclude “no agent.” That outcome is evidence of disciplined design, not a failure of ambition.

A selection that survives scrutiny keeps the doubt about autonomy on the table for the whole review — the possibility that it is an expensive substitute for product clarity — and settles it with a measurement rather than a preference. Why the measurement cannot be replaced by the preference is documented. METR ran a randomized controlled trial in 2025: 246 real tasks, 16 experienced open-source developers, each with an average of 5 years' experience on the repositories they were working in. Before starting, the developers forecast a 24% speedup. Afterwards, having lived through it, they estimated a 20% speedup. The measured result went the other way: “When developers are allowed to use AI tools, they take 19% longer to complete issues”.

These were not novices guessing about an unfamiliar tool. They were experts on their own repositories, reporting on work they had just finished, and their estimate was wrong by roughly forty points of direction. If a practitioner's own after-the-fact judgement of 246 completed tasks can point the opposite way from the stopwatch, a design review's judgement about an architecture nobody has run yet is worth less still. Collect the number: what the simpler baseline actually achieved against the success rate the task requires.