The Pulse
Microsoft Opens Agent Lightning, a 3,500-Line RL Framework
Agent Lightning v1.0 connects reinforcement-learning training to agents’ existing software harnesses through a model-request proxy. Microsoft reports a 14.6-point improvement for Qwen3.5-9B on SWE-bench Verified using about 6,000 training s
AI.info Team · Written with AI ·
Microsoft Research published its announcement about Agent Lightning v1.0 on October 7, 2026. The project’s GitHub repository records the open-source release in August 2026, presenting a roughly 3,500-line framework for training AI agents with reinforcement learning while keeping their existing software harnesses in the loop. The release targets a practical mismatch: agents rely on surrounding code to manage tools, context and execution, but many training systems require developers to rebuild that machinery inside the training framework. Microsoft’s researchers say Agent Lightning lets an agent keep its harness and connect to training through a model-request proxy.
“We present Agent Lightning v1.0, a lightweight framework for harnessed agentic RL implemented in approximately 3,500 lines of code.”
Zhiyuan He et al., authors of the technical report
Training through the agent’s existing harness
In a conventional agent reinforcement-learning setup, the training system typically owns the environment interaction loop. The model produces an action, receives an observation and continues from a growing token history. That approach becomes awkward when a deployed agent’s harness independently manages prompts, tools, context summaries, subagents and handoffs.
Agent Lightning places an OpenAI-compatible language-model proxy between the harness and the model. The harness continues to make its usual requests; the proxy records the prompts and responses used for training. Microsoft says developers can often connect an existing agent by changing its model endpoint rather than rewriting its control logic.
The paper calls this setup “harnessed agentic RL”: the deployed harness, rather than the trainer, controls interaction with the environment. That keeps the training setup closer to the agent developers intend to run, while leaving the training system to work from model request-and-response sequences.
Three components handle requests, rollouts and training
Microsoft describes the framework as a control plane made up of an API Gateway, a Rollout Controller and a Customized Trainer built on verl. The gateway proxies model calls and stores rollouts, models and events; the controller launches agents as local processes or Kubernetes jobs; the trainer assembles the resulting samples and updates the model.
Keeping those jobs separate matters because the harness can turn one agent run into a variable number of training samples. The authors identify four technical problems: text can be retokenized differently from the sampled tokens, subagents and context summarization can change sample counts, sample-level calculations can give some rollouts more weight, and a fixed GPU configuration must schedule workloads whose size becomes clear only after execution.
For those challenges, the team uses rollout-level reward and advantage calculations and rollout-level loss normalization. The framework also includes “Collocated Async RL,” which shares GPUs between agent rollouts and model updates. Microsoft reports about a twofold end-to-end speedup over synchronous RL in its experiments; that comparison is the team’s reported result, not a general performance guarantee for other workloads.
Qwen3.5-9B gains 14.6 points on SWE-bench Verified
The release includes a coding-agent training pipeline built with Qwen3.5-9B, mini-SWE-agent and the open-source SWE-smith dataset. The researchers report that reinforcement learning lifted Pass@1 on SWE-bench Verified from 41.8% to 56.4%, an absolute gain of 14.6 percentage points, using about 6,000 training samples.
The pipeline includes data cleaning, environment setup and safeguards against reward hacking, along with training scripts. Microsoft calls the compute requirement modest but does not give a single hardware cost or runtime figure in its announcement. The results therefore show what the team achieved with its recipe, not what every user should expect from a different model, harness or compute setup.
September’s coding-agent example uses a separate model
The project’s repository lists a separate September 2026 coding-agent example using the mixture-of-experts model Qwen3.5-35B-A3B. It reports a SWE-bench Verified increase from 47.8% to 61.6% after training on 1,800 examples. Those figures describe that later example, not the Qwen3.5-9B experiment detailed in the October 7 announcement.
Agent Lightning v1.0 is available under the MIT License, and its documentation includes examples for search, math and coding agents. The release’s clearest test is whether the proxy-based approach can make training practical without duplicating each agent’s harness—and whether researchers can reproduce the reported coding results with the released pipeline.