AutoSFT: An Agentic System for Automated Data Research in Supervised Fine-Tuning

AI summary

Overview

Research area: Automated AI research agents, specifically automated curation and construction of supervised fine-tuning (SFT) data for large language models.

Technical level: Intermediate. The paper assumes familiarity with supervised fine-tuning, instruction-following benchmarks, and LLM agent scaffolding, but its core arguments are about workflow structure rather than optimization internals.

Scope: The paper diagnoses why a single LLM agent collapses onto one kind of data edit when running an SFT data-research loop, and introduces AutoSFT, a multi-agent system whose separated execution, multi-angle analysis, and data synthesis components are each validated by retraining from the base checkpoint every round.

What This Paper Is About

When LLM agents are given control of the whole post-training stack, they consistently choose supervised fine-tuning and then spend most of their effort revising training data — but existing agent systems hand them only one lever, either selecting rows from a fixed pool or calling a generator. Real data engineers do much more: they change category proportions, prune and repair rows, replace bad responses, write examples for missing capabilities, and order the data.

The paper defines this broader space of six data-action families and asks whether an automated system can reach the full range rather than a single action, and what structure that requires. A pilot study shows a single agent reaches only one family and repeatedly loses its working context, motivating AutoSFT, a meta-agent-plus-subagents design evaluated on instruction following (IFEval, IFBench) and code generation (MBPP+).

Key Contributions

1. A data-action space for SFT research. The authors formalize six kinds of change a data engineer makes — adjusting category proportions, pruning rows, repairing rows, replacing overlong responses, writing examples for missing capabilities, and ordering the data — each fixed by what happens to rows and by the level at which the decision is made (category, row, capability, or whole recipe). Actions are labeled from the recipe diff after a run, not from the agent's stated reason.

2. A diagnosis of single-agent failure. In a corrected reproduction of the PostTrainBench setting (Qwen3-1.7B-Base, IFEval, six corrected design choices), one Codex agent running GPT-5.5 at medium reasoning effort submitted 33 rounds and completed 32 in 11.5 hours, staying entirely inside the mixture family while its context filled to 94.7% of a 258,400-token window and was compacted ten times.

3. AutoSFT, a multi-agent research policy. A meta agent owns each round and delegates to an executor that runs training/conversion/evaluation separately and returns artifact paths, four analysis subagents that read the same round from complementary angles (training data, failure cases, learning dynamics, history), and a synthesis subagent that writes data for undersupplied capabilities and is gated by verifiers.

4. Component-by-component attribution. Every comparison fixes the base model, initial pool, token budget, training script, and evaluation, and retrains from the base checkpoint each round, so each added component's effect is individually attributable rather than confounded with accumulated training.

Main Findings

• A single agent collapses to one action family. All 33 rounds of the pilot run fall in the mixture family (32 rounds labeled reweight, one replay). The agent has vocabulary for the move — 22 distinct expressions across the run — but no variety in the move itself. Round 9 is best on both benchmarks and the 23 rounds after it produce nothing better.

• Wall-clock pressure silently shrinks the training data. Under a ten-hour budget, the per-round recipe expands to 264k rows over the first four rounds and then declines every round, ending at 69k rows, or 26% of the peak. The agent never terminates early; it progressively reduces the cost of each remaining round. Under benchmark targets rather than a clock, this contraction does not occur.

• Context is consumed by execution output, not by data reasoning. Training output (13.4%), repeated reads of the run record (11.8%), applying the data action (9.7%), and evaluation output (8.9%), together with the agent's reasoning (16.3%) and compaction summaries (8.3%), take two thirds of the window. Polling job status is the most frequent operation at 371 calls but only 1.7% of the 9.9M bytes accumulated.

• The executor subagent halves context loss. Over comparable elapsed time, the base run compacts ten times and the executor run five. Shell commands fall from 58% of recorded events to 29%, and a third of what remains is delegation to the executor. The base agent issued 268 status checks, 23.0% of its commands.

• Analysis subagents raise IFEval and broaden the action mix. The analysis run reaches 63.33 at R6 — already above the best the base run attains in 50 rounds — and climbs to 67.10 at R30. The base run's best is 60.60 within the first 30 rounds and 62.35 across all 50, so analysis leads by 6.50 points at equal rounds and by 4.75 even when the base run is given 20 more rounds. Repair and resupervision together account for 55% of actions taken, versus a base run confined to mixture.

• Unlocked actions often set new bests. Selecting data by validation results sets a new best in 41.7% of its uses, and editing both instructions and answers in 6 of its 14 uses.

• Synthesis improves both instruction-following benchmarks. The synthesis run reaches a best IFEval of 81.58 versus 78.15 for the base, and a best IFBench of 51.95 versus 49.02. Every IFEval checkpoint from R9 onward exceeds the best checkpoint the base run attains over 31 rounds.

• Synthesis helps unevenly across capabilities. The repeat_prompt capability sits near 40% for the first 8 rounds, and the batch supplied at R9 raises it to 75.6% in the same round. By contrast, odd_even_syllables is targeted in 3 separate rounds and scores zero every round, suggesting supply alone cannot teach a constraint the model cannot yet follow.

• The structure transfers to code. On MBPP+ with Qwen3-1.7B-Base and a 742,842-example Python pool, the single agent shows no sustained improvement, Base + Analysis produces a consistent upward trend and reaches 64.8% at round 13, and adding synthesis reaches 65.8% by round 6 — a gain of +3.7 MBPP+ over the single agent, in fewer rounds despite extra runtime per round.

• A repeated round measures run-to-run noise. Round 15 accidentally reproduced round 9's exact recipe and scored IFEval 66.23 (−1.53) and IFBench 37.49 (−2.46), giving a concrete estimate of how much a repeated round moves.

Methodology in Plain English

The authors first define what "data research" means for SFT: a loop where the base checkpoint is fixed, and only a candidate pool of examples plus a "recipe" (which rows to train on, how often, in what order, and whether any row is edited or newly written) carries history between rounds. Training always re-initializes from the same base model, and every recipe within a comparison is capped at the same token budget, so two runs differ only in which rows they trained on — no agent can buy extra rounds with less data.

They then run this loop with a single agent to see what breaks, deliberately correcting six confounds in the original benchmark setting: fixing the algorithm to SFT, supplying an initial pool instead of internet search, adding IFBench as a second benchmark, retraining from base each round, replacing a wall-clock allowance with benchmark targets, and returning richer feedback (training loss, category-level scores, per-example failures). Each failure they observe maps to a structural requirement.

AutoSFT answers those requirements with role separation. An executor handles the entire job lifecycle — submission, monitoring, checkpoint conversion, evaluation — in its own session and returns only file paths and a short status report, so verbose logs never enter the deciding agent's context. Four analysis subagents read the same round through different lenses and each proposes one executable data action. A synthesis subagent derives capability targets, writes examples against each target's programmatic verifier, and admits them only after passing that verifier, an independent semantic review, and an n-gram overlap check against the evaluation sets. The meta agent integrates the proposals into the next recipe.

Two design details support attribution and honesty. The system maintains a memory keyed by round that records, before training, the hypothesis a round is testing — the planned change, why it should move the metric, the comparison round, the expected direction, and the finer observation that would show it working — and refuses submissions without one. Verdicts are read on two separate questions: what the benchmark mean did relative to the comparison round, and whether the finer observation actually appeared.

Why This Matters

Impact on research. The paper reframes the bottleneck in automated data research. The headline claim is not that any single data operation is powerful, but that coverage of the action space is what matters, and that multi-agent decomposition is a practical way to obtain it. Because every result comes from runs that retrain from base each round, the component ablations are unusually clean for an agentic system paper — a methodological contribution independent of the specific system. The finding that wall-clock budgets cause agents to quietly shrink their own training data is a warning for anyone evaluating agents under time limits.

Real-world applications:

• Cost-efficient fine-tuning pipelines. Teams with a fixed compute budget can use the executor/analysis split to keep a curation agent's context on data decisions rather than on job logs.
• Domain adaptation where no capability taxonomy exists. The MBPP+ experiment shows the meta agent building its own capability grouping from benchmark failures alone, which is the common case outside instruction following.
• Diagnosing capability gaps in proprietary models. The repeat_prompt and odd_even_syllables traces illustrate a workflow where a loop first identifies an undersupplied constraint and then either fixes it with targeted data or reveals that data alone will not.
• Benchmark and leaderboard engineering. Because the system optimizes reported scores directly, the paper's verification gate and off-target reporting are a template for guarding against data contamination when synthesis is automated.

Industry relevance. Post-training data curation is the dominant cost center in adapting open base models, and the paper's evidence that a curated recipe can be improved at a fixed token budget — with the trainer, model, and evaluation held constant — maps directly onto industrial fine-tuning operations. The system is released as open code, and the run ledger protocol makes results auditable.

Future Directions

• Capabilities that data supply cannot teach. odd_even_syllables scores zero in all 3 rounds that target it. What should the loop do when synthesis repeatedly fails — change the training objective, restructure the task, or abandon the target? The paper does not report a mechanism for this.

• Action families left unexplored. Even with analysis, neither run writes a new example (that is reserved for synthesis) and neither alters training order because SFT fixes a shuffling seed. Whether ordering actions are reachable or useful in this loop is not established.

• The attribution versus depth trade-off. From-base retraining makes every component attributable, but it also means no round inherits anything from an earlier one — the paper notes that a request to replay round 9 had to be served by reassembling a recipe from the same pool. How much performance is left on the table by refusing to accumulate training across rounds is not reported.

• Runtime cost of synthesis. In the code domain, Base + Analysis + Synthesis completes 6 rounds in 64 hours against 15 for the other two configurations, reaching a comparable peak in fewer rounds. Whether that trade is favorable at larger scale, and how the approach behaves beyond Qwen3-1.7B-Base and the two domains tested, are open questions.

Target Audience

Researchers and engineers working on LLM agents for AI research automation, and practitioners responsible for post-training data curation who want a structured account of which data operations exist and how to make an agent cover them. It is also useful for benchmark designers: the pilot study's six corrections to an open-ended agent evaluation, and its demonstration that wall-clock budgets distort per-round data volume, are directly transferable to how agent research benchmarks are specified. Readers need comfort with SFT and instruction-following benchmarks but no background in multi-agent systems design.

Authors’ abstract

Language model agents increasingly automate post-training, but when given the full training stack they converge on supervised fine-tuning and spend most effort revising training data. Existing systems give the agent one kind of control, selecting rows from a fixed pool or calling a generator, whereas a human data engineer adjusts proportions, prunes, repairs, replaces and synthesizes the data. We design a data action space for these changes and ask whether an automated system can make the full range rather than one of them, and what structure that requires. We first run this loop with a single agent, and it reaches only one action, adjusting category proportions, while its context fills with execution output and is discarded mid-search. We therefore introduce \textbf{AutoSFT}, a multi-agent system where a meta agent separates execution from analysis, gathers proposals from independent angles, and calls a synthesis subagent for capabilities the pool cannot cover, with the model, the trainer, and the budget fixed within each comparison. Adding each component in turn helps: the executor halves context loss, the analysis subagents lift the best IFEval by $+6.5$, and synthesis adds $+3.4$ IFEval and $+2.9$ IFBench. The same loop transfers to code, where analysis adds $+3.7$ MBPP+ over a single agent that does not improve. Every result comes from runs that retrain from the base model each round, so each component's effect is individually attributable. Our findings suggest that automating SFT data research depends more on covering diverse data actions than on a stronger single action.