Human–Agent Teams, Escalation, and Mixed Initiative

Mixed-initiative is not a loose adjective. The term arrived in 1999 with twelve numbered conditions attached to it: “critical factors for the effective integration of automated services with direct manipulation interfaces”. Eric Horvitz wrote them. Four of the twelve are the whole design surface of this lesson: (5) employing dialog to resolve key uncertainties, (6) allowing efficient direct invocation and termination, (7) minimizing the cost of poor guesses about action and timing, and (9) providing mechanisms for efficient agent−user collaboration to refine results.

Factor (9) carries one sentence worth reading twice: “We should design agents with the assumption that users may often wish to complete or refine an analysis provided by an agent.” Refinement by the human is a starting assumption of the design, not a courtesy added at the end.

A human presence does not automatically provide oversight. Reviewers need enough time, enough skill, the evidence in front of them, and organizational permission to disagree. They also need to know who holds the decision at each moment. And they need to know what happens when two authorities give opposite instructions.

The cost of not knowing is on the record. On 1 July 2002 at 21:35:32 hrs, near Ueberlingen over Lake Constance, a Tupolev TU154M and a Boeing B757-200 collided in mid-air. Seventy-one people were on board and none of them survived. The immediate cause was a conflict of initiative. The BFU, the German Federal Bureau of Aircraft Accidents Investigation, set it out in its 2004 report AX001-1-2/02: “The TU154M crew followed the ATC instruction to descend and continued to do so even after TCAS advised them to climb. This manoeuvre was performed contrary to the generated TCAS RA.”

The BFU did not stop at the cockpit. Among the systemic causes it listed the ACAS/TCAS regulations issued by ICAO, national authorities, the manufacturer and the operators: they “were not standardised, incomplete and partially contradictory”. Two initiatives, two instructions, and no rule that said which one wins.

Four things turn a human presence into oversight: role allocation, a shared situation view, an intervention point, and an escalation path. A fifth, the learning loop, is what keeps corrections from evaporating.

Role allocation assigns tasks according to capability, consequence, and legitimacy. The shared situation view exposes state, evidence, uncertainty, and pending effects. It means the reviewer sees what the agent saw, at the moment it decided. The 2012 review gives two testable levers on that view: the position of advice on screen, and providing information rather than a recommendation. The intervention point allows correction before irreversible action — a question about seconds and about ordering, not about intent.

The escalation path routes ambiguity, conflict, and exceptions to the right person. Ueberlingen is what an escalation path looks like when the rules governing it are “not standardised, incomplete and partially contradictory”. The learning loop captures human corrections without turning every decision into permanent policy. Give the escalation path and the learning loop an owner each and a test each.

An automated test vehicle struck and killed a pedestrian in Tempe, Arizona on 18 March 2018. The automated driving system detected her 5.6 seconds before impact, but “never accurately classified her as a pedestrian or predicted her path”. The safety operator was looking toward her cell phone at the bottom of the centre console. She redirected her gaze to the road about 1 second before impact. She began steering left 0.02 seconds before the vehicle struck the pedestrian at 39 mph. The car was an Uber Advanced Technologies Group test vehicle, and the National Transportation Safety Board adopted its report on the crash, NTSB/HAR-19/03, in November 2019.

The design had placed a person in the loop and given her a task the Board says people cannot perform. Its analysis of vehicle operator monitoring is blunt: “When it comes to the human capacity to monitor an automation system for its failures, research findings are consistent—humans are very poor at this task.”

Oversight is real when humans can identify and correct errors before consequential action, and not merely when a human name appears on the record. Three further signals say whether that first one holds in practice: reviewers see the evidence needed to disagree with the agent; escalation reaches people with appropriate authority and expertise; and human corrections improve the system through a governed feedback process.

Test each of them on repeated runs and on cases built to be misleading, not on a single clean example. Green and Chen tested theirs across every presentation condition they could construct, and every condition failed the same way.

Redesign the moment where a person already signs off on agent output, in a process you actually run. Ask first whether that sign-off is real review or a rubber stamp, since human review can hide a system that is impossible to review meaningfully.

Allocate roles: interpretation, evidence gathering, decision, execution, and appeal. Write down which authority wins when two of them disagree. The BFU found that rule missing at Ueberlingen.

Choose the intervention timing. Review goes before material commitment and after enough evidence exists. State the interval in seconds or hours. Tempe's was 5.6 seconds from detection and 0.02 seconds from the operator's first input.

Design the evidence view: source, uncertainty, alternatives, and expected consequence. Treat the position of advice on screen, and the choice to provide information rather than a recommendation, as the levers the 2012 review found they are.

Support correction. Allow edits, rejection, escalation, and rollback without workarounds — that is Horvitz's factor (6), allowing efficient direct invocation and termination.

Monitor team behavior: workload, override, rubber-stamping, delays, and repeated agent errors. Workload, task complexity and time constraint are the three mediators that review identified, and all three are yours to set.

Then change what the reviewer is given until they could plausibly catch a wrong output and stop it before it takes effect. Keep the record of the cases where they did.

The goal is not maximum autonomy or maximum review. It is a division of labor that improves outcomes while preserving accountability and recourse. The pooled evidence says that division is usually got wrong. A 2024 meta-analysis in Nature Human Behaviour pooled 370 effect sizes from 106 experimental studies published between January 2020 and June 2023. Its headline finding: “First, we found that, on average, human–AI combinations performed significantly worse than the best of humans or AI alone (Hedges' g = −0.23; 95% confidence interval, −0.39 to −0.07).”

The moderator matters more than the average. Decision tasks — approve or deny, release or hold, flag or pass — showed a significant loss (g = −0.27, 95% CI −0.44 to −0.10, P = 0.002). Content-creation tasks showed a positive but statistically non-significant effect (g = 0.19, 95% CI −0.09 to 0.48, P = 0.180) that was nonetheless significantly better than the decision-task result. The combination is worst precisely where this lesson's workflows live.

So before adding another checkbox to the workflow, ask what the last one actually caught. A review step nobody can perform properly buys the appearance of oversight and none of the substance, and on decision tasks it can cost you the performance of whichever party was better alone. The arrangement is worth building only where the person in the loop can still see enough, and act early enough, to stop a wrong action before it lands.