Humanize MLE Agents: Multi-Agent Session Alternation Beyond Single-Agent Limits

AI summary

Overview

Research area: LLM-based multi-agent systems for long-horizon machine learning engineering (AutoML-style autonomous agents), combining reinforcement-learning theory (semi-Markov decision processes) with empirical agent benchmarking.

Technical level: Intermediate to Advanced — the workflow design and empirical results are accessible, but the core argument rests on option/semi-MDP formalism, theorems, and budget bounds.

Scope: The paper diagnoses why single-agent MLE workflows degrade over long runs, formalizes alternating agent sessions as a context-reset semi-Markov decision process (CR-SMDP), and introduces HMA, a two-agent alternation workflow evaluated on MLE-bench and a kernel-optimization case.

What This Paper Is About

LLM agents that do machine learning engineering improve a solution by running repeated experiments (write code, run it, evaluate it) over many hours. The authors find that single agents slow down as a session drags on, and that custom "harnesses" built to guide an agent lose their advantage when the underlying model is upgraded. Their goal is to show that alternating two complete model–harness agents on a shared workspace, with the context reset at each handoff, can push performance beyond what either single agent reaches on its own.

Key Contributions

1. Identification of two single-agent failure modes. The paper names and measures long-horizon deceleration (reasoning effort per response declines as score gains shrink) across six models, and harness staleness (relative benefits of three custom MLE harnesses diminish or reverse as the backbone model improves), including one comparison on a 16-task MLE-bench subset.

2. A formal model of alternating sessions. Agent sessions are modeled as options in a context-reset semi-MDP (CR-SMDP), where handoffs preserve the shared workspace and retained result but reset context to empty. The analysis establishes sufficient conditions under which alternation exceeds single-agent reference limits (Theorem 3.3), derives an explicit sufficient experiment-budget bound (Corollary 3.4), and characterizes when fixed-k interruption preserves the advantage (Theorem 3.5, Corollary 3.6).

3. The HMA workflow. HMA alternates two native agents (unchanged tools, context management, and execution loops) on a shared workspace, caps each session at at most k experiments, resets context between sessions, and reserves the final t_rev minutes for the other agent to review the archive of candidates.

4. State-of-the-art MLE-bench results with cross-pair generality. HMA reaches a 78.2% any-medal rate on 75 MLE-bench tasks, and across six configurations spanning five heterogeneous agent pairs it improves any-medal rate over constituent baselines by 6.4 points on average.

Main Findings

• Long-horizon deceleration is widespread. Across six single-session /goal configurations on 75 tasks (responses pooled across tasks and repeats), mean output tokens per response — used as a proxy for reasoning intensity — fall by 29.8%–45.0% from hours 0–2 to hours 3–5.

• Harness benefits fade as backbones improve. On a 16-task MLE-bench subset comparing DeepSeek-V4-Flash and DeepSeek-V4.1-Flash: the DeepSeek native harness goes from 16.7% to 77.1% medals; ScienceFlow (24h) goes from 25.0% (+8.3 over native) to 68.8% (−8.3); ML-Master 2.0 (24h) from 6.3% (−10.4) to 37.5% (−39.6); MLEvolve (12h) from 18.8% (+2.1) to 50.0% (−27.1). The gap persists despite the specialized workflows' longer execution budgets.

• HMA sets a new any-medal mark. Opus-first HMA reaches 78.2% overall medals versus 72.4% for the strongest single-agent baseline measured in the paper and 70.2% for the highest published workflow baseline — an 8.0-point gain over the latter while using a 6-hour budget, where published workflows use 12–24 hours.

• HMA improves every heterogeneous pair tested. Over five model pairs with constituent medal rates of 54.2%–72.4%, all six HMA configurations beat the stronger constituent in overall medal rate. Signed gains over the average of the two constituents range from −1.5 (Lite tasks in the GPT–GLM pair; overall +5.6) to +9.6 overall (GPT–Kimi). The reported average improvement is 6.4 points.

• Starting order is not decisive. For the Opus–GPT pair, both orders beat the stronger constituent's 72.4%: GPT-first reaches 73.3%, Opus-first 78.2%.

• HMA sustains reasoning effort. From hours 0–2 to hours 3–5, HMA retains 86.4% of its mean output tokens per response, versus 60.7% for matched single-agent goal references. The six HMA configurations reach 42.7%–60.0% medals within two hours and keep improving through six hours.

• Short interruption caps preserve most gains. Across eleven configurations, k = 5 retains 97.83% of observed gains with a mean within-option experiment retention of 73.77%, motivating candidate caps of k ≤ 5.

• Final review improves medal retention. Holding exploration fixed, final review changes 8.9% of selections and raises any-medal outcomes from 311 to 319 with no losses. Of 450 task runs, 330 produce at least one medal-winning exploration candidate; final selection retains a medal in 319 (96.7%) of those runs, versus 311 (94.2%) before review.

• A cross-model MLE case shows continued refinement. On INGV Volcanic Eruption Prediction, GPT and Opus improve the historical best MAE in five of six post-handoff options; the final MAE of 34,234 is 46.5% below the best of six constituent goal runs, and mean response length rises on four of five returns to a previously used model.

• A kernel-optimization case extends beyond MLE. On Anthropic's Original Performance Take-Home (AOPT), Claude Opus 5.5 and GPT-6-astra alternate with a cap of three accepted submissions per session; the selected 48-hour run reaches 878 cycles at 34 hours 33 minutes and keeps it in the final candidate — 14 cycles (1.62%) above the leading score of 864 cycles and matching the 10th-ranked score on the Paradigm community leaderboard as of October 8, 2026. The paper notes this is a comparison of reported scores, not a verified leaderboard placement, because that leaderboard uses a separate nine-seed judge.

Methodology in Plain English

The authors start empirically. They run single agents for long sessions, track how many output tokens each response takes, and show effort fading as scores plateau. They then compare off-the-shelf custom MLE harnesses against the official native harness on the same tasks using a weaker and a stronger version of the same model family, and show the custom harnesses' relative edge disappearing with the stronger model.

To explain why alternation could help, they borrow the options framework from reinforcement learning. Each agent session is treated as an "option" with an initiation condition, an internal policy, and a termination rule; the state consists of the shared workspace, the retained best result, and the active agent's context. Handing off resets only the context, keeping the workspace and result. Under two assumptions — that the best result is never lost, and that each agent can persistently reach useful states for its partner — they prove that the alternation's quality converges above the ceiling achievable by any single-agent workflow in the reference family, with the failure probability shrinking geometrically in the number of cycles. They then derive how many experiments are needed to hit a target success probability, and analyze whether forcibly interrupting a session early (fixed k) hurts.

In practice, they set k = 5 based on the empirical finding that short prefixes retain nearly all observed gains. The HMA loop is simple: run one agent for at most five experiments, preserve the workspace and candidate archive, reset context, switch to the other agent, repeat until the time budget minus a review window is exhausted, then let the other agent review the archived candidates and pick the final answer. Agents keep their native harnesses, run at max reasoning effort, receive no task-specific knowledge at initialization, and each task run gets one NVIDIA A10 GPU under a six-hour budget with the final 15 minutes reserved for review.

Why This Matters

The paper reframes multi-agent MLE as a workflow-coordination problem rather than a harness-engineering problem. If improvements come from alternation itself, they should transfer across model generations — directly addressing the "harness staleness" problem where custom scaffolding is invalidated by the next model release. The CR-SMDP formulation also gives a reusable theoretical language (reference ceilings, interruption conditions, budget bounds) for reasoning about when cooperation between agents can beat the best individual.

Real-world applications implied by the work:

• Automated ML pipelines: producing competition-grade or production-grade models on Kaggle-style tasks without a dedicated ML engineer driving each iteration.
• GPU and systems kernel optimization: the AOPT case shows the same alternation loop applying to low-level performance tuning measured in execution cycles.
• Enterprise agent fleets: organizations already running agents from multiple vendors (OpenAI, Anthropic, Google, DeepSeek, and others) can coordinate existing agents instead of rebuilding custom scaffolding per model upgrade.
• Long-running research and engineering tasks: any iterative loop where artifacts persist — code, trained models, evaluated candidates — and where effort provably decays over long single-agent sessions.

Industry relevance is concentrated where model churn is fastest: teams that maintain bespoke agent harnesses absorb re-tuning costs at every model upgrade, and this paper offers an alternative that leaves native harnesses untouched.

Future Directions

• Establishing comparative gains beyond MLE-bench. The paper explicitly states that the 48-hour kernel-optimization case is preliminary single-run evidence and that "establishing comparative gains beyond MLE-bench remains future work."
• Choosing k systematically. The cap of k = 5 is selected empirically, and Corollary 3.6 gives only a qualitative bound (1 ≤ k ≤ k₀) under an assumption that interruption losses decay geometrically with k. Tighter theoretical guidance and better automatic cap selection are open.
• Extending beyond two agents. The analysis is built for a pair (A, B) with a fixed alternation policy; whether larger or asymmetric teams, or learned scheduling, improve on the pairwise guarantee is untested.
• Understanding when cooperation assumptions hold. The guarantee depends on Assumption 3.2, which requires each agent to persistently reach states useful to its partner. Characterizing which model pairs satisfy this, and predicting pair compatibility in advance, remains open.

Target Audience

Researchers and engineers working on LLM agents for machine learning engineering, multi-agent orchestration, and long-horizon autonomous systems will get the most from this paper. It is also relevant to reinforcement-learning researchers interested in options and semi-MDPs applied to LLM workflows, and to practitioners who maintain custom agent harnesses and face re-tuning costs with each model upgrade. Readers wanting only the empirical takeaway can focus on Sections 2 and 4; the theoretical contribution requires comfort with options, Bellman recursions, and convergence arguments.

Authors’ abstract

Existing LLM-based agent approaches for long-horizon machine learning engineering (MLE) often use context engineering, implemented through custom harnesses that manage tools, skills, and accumulated experience. However, we find that as single-session execution lengthens, the agent's reasoning intensity declines alongside slowing progress, summarized as long-horizon deceleration, and the benefits of these custom harnesses vanish as the underlying model improves, summarized as harness staleness. These findings suggest that their benefits may not persist across versions, which motivates a workflow-level approach in which agents build on one another's work across sessions. We model these alternating agent sessions as a context-reset semi-Markov decision process (CR-SMDP) and establish sufficient conditions for alternation to exceed single-agent reference limits, derive an explicit experiment-budget bound, and characterize when session caps preserve this advantage. Guided by this analysis, we introduce Humanize MLE Agents (HMA), which alternates two agents on a shared workspace, limits each session to at most k experiments and resets the context between sessions, without additional task-specific harness or prior knowledge supplements. Across 75 Kaggle tasks in MLE-bench, HMA reaches a state-of-the-art any-medal rate of 78.2%, exceeding the strongest open-source baseline by 8.0 points with a quarter of its time budget. Moreover, among six configurations spanning five heterogeneous agent pairs, HMA improves any-medal rate over the constituent baselines by 6.4 points on average, demonstrating cross-pair generality. A supplementary case on kernel optimization reaches 878 cycles, within 1.6% of the community leaderboard leader, demonstrating applicability beyond MLE.