Transactions, Side Effects, and Compensation

τ-bench refuses to take an agent's word for what it did. It scores the agent against the end state of a database, not against its final message. Yao and Shinn published it with two co-authors in 2024. Its airline domain holds 500 users, 300 flights and 2,000 reservations. The agent gets 6 write tools and 7 non-write tools, across 50 tasks. gpt-4o with function calling completes 35.2% of the airline tasks and 61.2% of the retail ones at pass^1. Run the retail tasks eight times each and pass^8 falls to roughly 25%. The scarce property is not capability. It is repeatability.

One documented failure has exactly the shape this lesson is about. A customer asks for two items to be exchanged, and the exchange tool can be used once. The paper says what the agent did with it: “However, the gpt-4o FC agent omits the domain knowledge and rule and decides to exchange one item first, resulting in the second item not being exchanged.” The unrecoverable step was spent on half the request. Nothing in the dialogue announces this. The transcript reads as a cooperative, competent exchange. Only the database disagrees.

Agent tasks often cross systems that cannot share one atomic transaction. The runtime must order effects, stage reversible changes, and define compensation for operations that can complete independently.

The vocabulary is not improvised. The saga and its compensating transaction were defined in 1987, in a paper by Garcia-Molina and Salem. Its abstract sets out the guarantee: “The database management system guarantees that either all the transactions in a saga are successfully completed or compensating transactions are run to amend a partial execution.”

Read the verb. Compensating transactions amend a partial execution. They do not repeal it. The paper's own airline example says as much outright. A compensating transaction undoes the earlier one only from a semantic point of view. It does not necessarily return the database to the state that existed when that transaction began, because other transactions may have run in between.

Compensation is not time travel. A refund may not restore inventory, reputation, privacy, or a missed deadline, so the task contract must expose residual harm. Residual harm is what the compensating step cannot give back: the room somebody else has now booked in that interval, the hour the traveler spent on the phone.

Check preconditions, stage or reserve what can be held, decide the commit order, verify each committed step. Verification and compensation need separate owners and separate tests. A hold is not a booking. It reserves the option and can be dropped, which is why it belongs before anything irreversible.

In one industry that staging step is not a design convention. It is a rule with a number. Covered US carriers have to offer the hold, and 14 CFR § 259.5(b)(4) says in what terms: “Allowing reservations to be held at the quoted fare without payment, or cancelled without penalty, for at least twenty-four hours after the reservation is made if the reservation is made one week or more prior to a flight's departure;”

The requirement was contested and it survived. Spirit Airlines argued that the twenty-four-hour cancellation requirement was arbitrary and capricious. The D.C. Circuit rejected that and denied the petitions for review on 24 July 2012. The Supreme Court denied certiorari on 1 April 2013. For a whole industry, then, the stage-or-reserve step of this sequence is twenty-four hours long and legally mandatory. That is a useful thing to know before designing a workflow that skips it.

Three systems, one task, no shared transaction. Design the saga for it, including what each compensating step cannot restore. Then read the sequence back in order. At each position, ask what would still be standing if the next call never returned.

That question separates a plan that merely reads well from one you can defend. An order can look reasonable end to end and still spend the unrecoverable step early. The τ-bench exchange is precisely that shape: a single-use tool applied to the first item, “resulting in the second item not being exchanged.”

Work against the definitions rather than against your own vocabulary. Mark each call as idempotent or not in the sense RFC 9110 gives the word. Mark the one Microsoft would call the pivot transaction, since after it the only remaining strategy is completing forward. For each step, note what you would keep afterwards as proof that it really committed. Then say plainly which steps have no compensating transaction at all.

Look first for the plainest proof: every committed step has a verification record. A reviewer should be able to state what happened, not infer it from what came later. A system that genuinely has such a record can describe it in public. Stripe does: “Stripe’s idempotency works by saving the resulting status code and body of the first request made for any given idempotency key, regardless of whether it succeeds or fails.” The record has stated dimensions rather than implied ones. Idempotency keys run up to 255 characters. Keys can be removed automatically once they are at least 24 hours old. The guarantee has a documented expiry.

The same mechanism has been taken to a standards body, and its status is part of the evidence. An IETF draft on the Idempotency-Key HTTP header field requires that the key MUST be unique and MUST NOT be reused with another request with a different request payload. Its seventh revision, posted on 15 October 2025, is still the latest. It expired on 18 April 2026 without being published as an RFC. So this is production practice with a draft behind it, not a ratified standard. A review should say which of the two it is relying on.

A separate question is whether partial completion produces a known state rather than an improvised recovery. A run that stops halfway should land somewhere the team has already named and can describe. Two further checks show themselves only under repeated or adversarial cases: that compensation limits are visible to users and operators, and that the most irreversible effect occurs only after prerequisite evidence is verified.

For high-impact workflows, the runtime should own transaction choreography. The model may select among approved branches. It should not invent compensation rules during an incident.

There is a regulator's account of what inventing one mid-incident costs. A stale code path left on one of eight SMARS servers turned 212 parent orders into over 4 million executions, in 154 stocks and more than 397 million shares. It ran about 45 minutes and cost $460 million. The SEC's 2013 order against Knight Capital Americas records all of it and carries a $12 million penalty. The recovery is the part to read twice. “In one of its attempts to address the problem, Knight uninstalled the new RLP code from the seven servers where it had been deployed correctly. This action worsened the problem, causing additional incoming parent orders to activate the Power Peg code that was present on those servers, similar to what had already occurred on the eighth server.” The improvised rollback was itself a compensating action. It took the blast radius from one server to eight.

Nobody should approve a workflow of this kind without walking its commit order once more, in sequence, from the first call to the last. A plan can be sensible in every individual step and still place the irreversible one ahead of the checks that were supposed to justify it. The second thing to have in front of you is the run record. It says what was committed, when, and what would undo it. Where it names a step that nothing would undo, that limit belongs in the design notes, not in an incident review.