Terminal-Universe: Turning Agent Trajectories into Scalable Terminal Environments

AI summary

Overview

Research area: LLM-based coding agents, specifically the construction of executable terminal environments for agent post-training (supervised fine-tuning).

Technical level: Intermediate. The paper assumes familiarity with coding agents, tool-calling trajectories, containers, SFT, and benchmark evaluation, but its core idea is explained through a simple inversion (environment to trajectory, versus trajectory to environment).

One-sentence scope: The paper presents Terminal-Universe, a framework that reconstructs reusable, verifiable terminal environments from recorded agent trajectories and then re-queries those environments to synthesize new training tasks.

What This Paper Is About

Terminal-based coding agents have generated large numbers of recorded trajectories, but the environments those agents actually ran in remain scarce. The authors observe that a trajectory is a single frozen demonstration whose correctness cannot be verified, while an environment can be re-queried, re-solved by a stronger model, and checked by tests.

The goal is to invert the usual mapping: instead of rolling out a trajectory from an environment, reconstruct an environment from a trajectory. The paper's framework replays the file operations in a trajectory to recover the workspace, fills in missing files with a completion agent, and then generates new tasks inside, across, and iteratively beyond that workspace.

Key Contributions

1. Environment reconstruction from trajectories. The paper reframes recorded agent trajectories as a source of reusable executable environments. Each one is reconstructed through deterministic replay of recorded file operations followed by agentic completion, without needing the original repository or building from scratch.

2. Four re-querying mechanisms. Using Intent Recovery to reconstruct the source task, Single-WS to synthesize new tasks within one workspace, Cross-WS for breadth via cross-workspace tasks over mined directional dependency relations, and Multi-Round for depth via multi-round user-agent sessions. Each task is paired with an agent-authored verifier, and only passing trajectories are kept.

3. An at-scale dataset and empirical validation. From public terminal agent trajectories the pipeline produces 37.3k task-sufficient environments and 32.0k verifier-filtered SFT records. Supervised fine-tuning of Qwen3.5-27B on this corpus improves Terminal-Bench 2.1 by 11.9 points and EvoCode-Bench v2 MT@4 by 13.8 points over the same base model.

4. Ablations isolating each design choice. The studies show that re-solving recovered tasks far outperforms imitating the raw trajectories, that agentic completion matters, that verifier filtering matters most for harder tasks, and that breadth and depth expansions each add supervision.

Main Findings

• Re-solving beats imitation. On Terminal-Bench 2.1, SFT on 35.8k source trajectories averages 36.7 (33.0 under Claude Code, 40.3 under Terminus2-XML), while Intent Recovery at the same volume averages 52.1 (51.3 and 52.9). The base Qwen3.5-27B scores 47.0 on average (47.8 and 46.2).

• Agentic completion is worth its cost. At matched volume (35.8k records), training on completed environments reaches 52.9 ± 1.4 versus 48.7 ± 3.5 for replay-only, a 4.2-point gap with lower variance. Replay-only still gains 2.5 points over the base model (46.2 to 48.7).

• Verifier filtering matters most for harder tasks. On Single-WS, verifier-passed data scores 56.4 on 25.4k records versus 56.0 on 35.1k unfiltered records. On Cross-WS, filtering raises the score from 53.2 on 7.1k records to 55.4 on 3.5k records.

• Cross-workspace data adds breadth. Cross-WS alone reaches 55.4 with 3.5k records; adding it to Single-WS raises performance from 56.4 to 58.4 on 28.9k records. Cross-WS trajectories are longer and harder: at the median, 23 assistant turns versus 14, 38 tool calls versus 20, and 46.5k tokens per record versus 30.4k, with teacher pass@1 dropping from 72.3% to 49.2%.

• Where breadth helps by category. On Terminal-Bench 2.1, software engineering, the largest category with 26 tasks, improves from 46.9 to 49.1, with larger gains in model training (+15.0) and debugging (+10.0). Three categories decline by 2.1 to 2.5 points, a range the authors note falls within run-to-run variation for categories of at most eight tasks.

• Multi-round data improves persistent-task performance. Adding Multi-Round data to Single-WS raises EvoCode-Bench v2 MT@4 from 18.4 to 21.0 and Case score from 71.9 to 76.9. Removing round-level verifier feedback drops MT@4 by 2.2 and Case score by 3.7, yielding longer but lower-quality trajectories.

• Reconstruction statistics. Replay initially leaves workspaces sparse: mean file count rises from 2.9 to 22.4 after agentic completion. Sufficiency after replay is 40.2% for Terminal and 20.1% for SWE workspaces, rising to 93.5% and 77.1% after completion, across 38,294 evaluated terminal environments and 1,900 SWE repositories. In total 37,273 fully sufficient environments are identified.

• Corpus and composition. The SFT corpus contains 31,977 demonstrations: 25,386 Single-WS, 3,512 Cross-WS, and 3,079 Multi-Round, totaling approximately 1.42B training tokens. The retained Multi-Round records average 4.51 rounds, and 69.6% contain a failure the session subsequently repairs. Python is the dominant primary language at 84.7%, with data processing, DevOps, and security workloads collectively accounting for over 80% of technical domains.

• Full mixture results. The Full Mixture (32.0k records) reaches 52.8% on Terminal-Bench 2.0 and 58.1% on Terminal-Bench 2.1 under Terminus2-XML (+11.2 and +11.9 over base), and 58.2% on Terminal-Bench 2.1 under Claude Code (+10.4 over base). On EvoCode-Bench v2 it raises MT@4 from 6.3 to 20.1 and Case score from 67.8 to 76.1. The teacher model Qwen3.7-Max scores 69.7 on Terminal-Bench 2.0, 74.5 on 2.1, 39.8 MT@4, and 83.4 Case score.

• Comparison against other construction methods. In Table 1, Terminal-Universe reports 37.3k environments and 32.0k tasks with verifier, multi-round, and cross-workspace support, whereas the compared methods (Endless Term., TMax, CLI-Gym, CLI-Universe, SkillSynth, OpenThinker-Agent, RST, CalibForge) report verifiers but not multi-round or cross-workspace support. In Table 3, among task synthesis methods, RST-27B reaches 49.4 on Terminal-Bench 2.0 and TMax-27B reaches 42.7 on 2.0, 44.9 on 2.1, 17.1 MT@4, and 72.5 Case score.

• Results not reported in the supplied content. The content provided is truncated inside Table 10, so the outcome of the expansion-axis ablation (more environments versus more queries versus more teacher solutions, all at roughly 35k records) is not reported here. Section 6.7, which tests whether the pipeline transfers beyond terminal workspaces, is announced but its results are also outside the provided content.

Methodology in Plain English

The authors start from a simple observation: when an agent works in a terminal, its tool calls leave a record. Read shows file contents, and Write and Edit show how files changed. That record is enough to rebuild a copy of the workspace the agent worked in.

Reconstruction has three stages. First, deterministic replay walks through the read, write, and edit operations in chronological order and restores each file to its earliest observed version, before the agent's first change. Files the agent created are excluded, and the agent's own edits are set aside for later verification. This produces a partial workspace, since the trajectory only exposes files the agent touched and file contents may be truncated.

Second, a completion agent fills in what is missing: it creates missing files, completes partial files, and restores dependencies so the task is solvable, without implementing the solution.

Third, an agentic judge inspects each completed workspace with read-only shell and file tools and labels it sufficient or insufficient based on whether the source, configuration, data, and structure give a capable agent enough context. Only sufficient workspaces continue. Every workspace runs in a standardized ubuntu:24.04 container with network access.

Each sufficient workspace is then re-queried in four ways. Intent Recovery reconstructs the original user request. Single-WS has an offline generator propose five self-contained candidate tasks per workspace, of which one valid candidate is randomly selected. Cross-WS profiles workspaces, retrieves candidate pairs via TF-IDF nearest-neighbor search, and uses an LLM judge to identify directional dependencies where a target workspace lacks a capability a reference workspace already implements; the task mounts the reference read-only at a separate path and gives the solver only the target behaviors and the mount path. Multi-Round introduces a user agent after the initial response, maintaining a requirement tracker of active, satisfied, and updated requirements, with round-level acceptance tests authored before the coding agent acts and failures translated back into natural user complaints. Sessions run up to six follow-up rounds or until a termination sentinel.

Every Single-WS and Cross-WS task gets an executable pytest verifier written by a dedicated agent inside the container. Solutions are rolled out with Qwen3.7-Max as the teacher (xhigh effort) in the Claude Code scaffold, at temperature 1.0 and top-p 0.95, over a 256k-token context with turn responses capped at 65,536 tokens, proactive summarization at 176k tokens, a maximum of 500 agent turns, and a four-hour wall-clock timeout. A trajectory is kept only if all tests pass; Multi-Round is selected at the round level, retaining a session only if it has at least two verified passing rounds while preserving intermediate failures that precede recovery.

Training is SFT on Qwen3.5-27B for two epochs with a constant learning rate of 7×10⁻⁶, a global batch size of 256, and a 256k-token sequence length, preceded by a 13-gram contamination check against Terminal-Bench tasks.

Why This Matters

Impact on research: The paper introduces a new axis for scaling agent training data. Rather than authoring environments by hand, mining git history, injecting bugs, or generating environments from scratch with a task, it treats trajectories that already exist at scale as indirect observations of environments. The ablation showing that re-solving recovered tasks (52.1 average) far exceeds imitating the original trajectories (36.7 average) is a direct result about where supervision value lies, and the supporting claim that the same metric can be re-queried by a stronger model and verified by tests addresses a verification gap that trajectories alone cannot close.

Real-world applications:

• Training and post-training of terminal or CLI coding agents, where the bottleneck is the supply of realistic, verifiable workspaces rather than demonstration data.
• Generating hard cross-codebase engineering tasks, such as reading a reference implementation, porting a feature between projects, or connecting two components across repositories.
• Producing multi-turn supervision for interactive coding assistants, where users refine requirements over a session and the agent must diagnose and recover from failures.
• Building or extending internal evaluation suites, since every synthesized task is paired with an executable verifier authored inside the container.

Industry relevance: The corpus is built from public terminal agent trajectories and uses a standardized ubuntu:24.04 container with network access, which the authors state lowers cost and simplifies deployment relative to repository-specific images, though they note prior work reports a modest reduction in resolve rate. The scale reported, 37.3k task-sufficient environments and 32.0k verifier-filtered records at approximately 1.42B tokens, is directly usable as an SFT corpus, and the reported gains of 11.9 points on Terminal-Bench 2.1 and 13.8 points on EvoCode-Bench v2 MT@4 over the same Qwen3.5-27B base model indicate the pipeline can be applied to existing agent checkpoints.

Future Directions

• Transfer beyond terminal workspaces. Section 6.7 announces a test of whether the pipeline generalizes past terminal environments, but the supplied content does not report its outcome.
• Budget allocation across expansion axes. The expansion-axis ablation (more environments versus more queries versus more teacher solutions at a matched ~35k-record budget) is truncated in the provided content, leaving the best allocation strategy unresolved.
• Improving replay-only reconstruction. Replay alone reaches 48.7 with ±3.5 variance, and the authors observe that the teacher often repairs missing files or setup before addressing the task, so some replay-only supervision targets workspace repair rather than the recovered task.
• Extending session length and interaction styles. Multi-Round sessions continue for up to six follow-up rounds, and the retained records average 4.51 rounds, with requests classified into feature extension, feature revision, and feature conflict; longer or differently distributed interactions are unexplored here.
• Closing the gap to the teacher. Even the Full Mixture reaches 58.1% on Terminal-Bench 2.1 under Terminus2-XML against 74.5% for Qwen3.7-Max, leaving substantial headroom.
• Narrowing the modest category regressions from cross-workspace data, where three Terminal-Bench 2.1 categories decline by 2.1 to 2.5 points, a range the authors attribute to variation for categories of at most eight tasks.

Target Audience

Researchers and engineers working on coding agents and agent post-training benefit most, particularly those building terminal or CLI agents, synthetic task generation pipelines, or multi-turn interactive agent training data. Practitioners responsible for curating SFT corpora will find the reconstruction and filtering statistics directly actionable. Benchmark designers and evaluation researchers will find the verifier construction and decontamination methodology relevant. Readers need working familiarity with tool-calling agents, containerized execution, and supervised fine-tuning; the central inversion of the trajectory-environment relationship is otherwise explained at an accessible level.

Authors’ abstract

As terminal-based code agents become prevalent, agent trajectories have accumulated at scale, while realistic, executable environments remain scarce. However, environments are what agent post-training actually requires: each can be re-queried into many verifiable tasks and provides execution feedback, whereas a trajectory is a single frozen demonstration. Rather than generating environments from scratch, we observe that the tool-execution history in existing trajectories exposes the structure and contents of the environments in which they ran, making it possible to reconstruct those environments from the trajectories themselves. Thus, we introduce Terminal-Universe, a framework which turns each trajectory into a reusable environment and explores it for synthesizing new tasks and continued interactions. Specifically, Terminal-Universe replays the file operations recorded in a trajectory to restore each file before the agent modified it, yielding a partial workspace; a completion agent then supplies the missing files and dependencies. On this recovered workspace, we both reconstruct the original intent task and synthesize entirely new ones. Besides, we also scale the tasks along two complementary axes: breadth and depth. For breadth, we mine directional dependency relations between related environments and synthesize cross-workspace queries spanning multiple codebases, as developers routinely do in real-world development. For depth, we extend the initial single-turn query into a multi-round session that captures iterative user feedback and requirement refinement via a user agent. Applied to public terminal agent trajectories, Terminal-Universe produces 37.3k task-sufficient environments. Supervised fine-tuning of Qwen3.5-27B on this corpus improves single-round performance on Terminal-Bench 2.1 by 11.9 points and multi-round performance on EvoCode-Bench v2 MT@4 by 13.8 points.