The Pulse
Harvard Publishes Agent Traces Spanning 209 Billion Input Tokens
Harvard MaDSys has released FreeInference traces covering 12,002 agent sessions, 1.19 million model requests and 1.21 million tool calls. The dataset preserves timing and cache metadata for research while omitting prompts, responses and too
AI.info Team · Written with AI ·
“Tool execution dominates active-session latency, yet tool parallelism remains uncommon.”
William Nixon, Muxin Tian, Yunjia Zheng, Haryadi S. Gunawi and Juncheng Yang, authors of “From Requests to Sessions”
Harvard MaDSys has released a record of how coding and assistant agents use language models: 12,002 sessions, 1,186,582 model requests and 1,213,347 tool calls gathered through its FreeInference gateway. The traces cover 16 weeks, from May 17 to September 5, 2026, and account for 209.2 billion input tokens. Rather than publish the conversations, the team offers timing, token counts, tool activity and cache-related metadata.
What the 209 billion tokens measure
The release captures requests from 267 accounts. Researchers reconstructed sessions from request histories; on the subset of agent tools that provide explicit session IDs, 95% of reconstructed sessions matched those IDs one-to-one. The paper describes 13 agent harnesses and lists a separate SWE-Bench replay in its trace statistics; the dataset card groups the release as 14 harnesses.
The 209.2 billion figure is cumulative input processed across requests, not a collection of 209 billion unique text tokens. The paper counts 197.1 billion cached tokens and 552.5 million output tokens. That distinction matters in agent sessions, where the same growing context can accompany many successive model calls.
Traces preserve structure, not conversations
Each session records an ordered sequence of model requests and tool calls, including request timing, serving latency, model name, anonymized provider, token usage and context changes. Inputs are represented as chains of 16-token blocks generated with the o200k_base tokenizer, giving researchers a way to study prefix reuse without seeing the text behind those blocks. Tool calls can include normalized categories, outcomes and latency; subagent sessions are nested under the calls that started them.
The release excludes message text, prompts, model responses, tool-result text and raw identifiers. Sensitive tool-argument values are replaced with typed placeholders, and rare tool names or commands used by fewer than three accounts are generalized. The team says the underlying study uses traffic from people who consented to research data collection and sharing. The dataset is released under the Creative Commons Attribution 4.0 license, with raw JSONL traces and flattened Parquet tables available through Hugging Face.
Cache reads dominate the paper’s cost model
The researchers find that requests carry large contexts but add comparatively little fresh input or generated output. In their analysis, the median request has roughly 129,000 input tokens, about 500 uncached tokens and about 200 output tokens. Under the paper’s modeled prices—$1 per million cached input tokens, $10 per million uncached input tokens and $50 per million output tokens—cached input makes up 72.9% of API cost among requests with reported cache usage.
The result describes that pricing assumption, not every provider’s bill. It does, however, show why counting only new prompt tokens can miss much of an agent session’s inference economics: large reused prefixes still carry a cost, even when their unit price is discounted.
Tool waits change the speed calculation
In the paper’s latency simulation, halving tool time produces a 1.38× session speedup, compared with 1.09× for halving prefill time and 1.16× for halving decoding time. The authors also report that 4% of requests that alter earlier context account for 62% of freshly prefilled tokens under an infinite-cache-retention assumption. Both results point to session behavior—not just isolated model calls—as a useful unit for testing serving systems.
The traces cannot reveal whether a context change was necessary or whether an agent completed its task well: the released data omits the words needed to judge either. Sessions that cross a week boundary are split in the release, and most reported findings come from one public inference service. Researchers can download the Parquet tables, reproduce the paper’s figures and examine the recorded requests, but the released files contain no readable agent conversations.