The Pulse
LightOn Releases ColGrep Agent With 34.3% Fewer Claude Code Tokens
LightOn released ColGrep-Agent-2B, a local code-search model that sends relevant file paths and line ranges to larger coding assistants. In LightOn’s 52-question Claude Code test, the agent reduced API token use by 34.3% but increased avera
AI.info Team · Written with AI ·
“We are releasing ColGrep-Agent-2B, an open 2-billion-parameter code-search agent that runs locally and sits between your coding agent and the repository.”
Maxence Lasbordes, Raphael Sourty, Aarush, and Amélie Chatelain, LightOn
LightOn announced ColGrep-Agent-2B on October 12, pitching it as a local search specialist for coding assistants rather than a model that writes or repairs code. The agent searches a repository, reads promising files and returns paths, line ranges and code so a larger model can begin with a narrower view of the project.
The release targets a common but costly stage of coding-agent work: locating where a behavior lives before deciding how to change it. ColGrep-Agent uses LightOn’s semantic code-search tool, ColGrep, alongside a read-only terminal. Developers can connect it to Claude Code, Codex or OpenCode; the larger assistant retains responsibility for interpreting the results and making code changes.
LightOn’s test cut tokens, not coding time
LightOn tested the tool with Claude Code running Sonnet 5.5 on 52 code-search questions across 13 public repositories. The company says the repositories were created in 2026 to reduce the chance the models had seen them during training, and the test covered Rust, Go, TypeScript and Python projects ranging from 17,000 to 1.8 million lines. Each question ran three times in each of four configurations, for 624 sessions.
With Claude Code delegating repository searches to ColGrep-Agent, API token use fell 34.3% against Claude Code’s built-in grep, find and file-reading tools: 93,000 tokens per question versus 142,000. LightOn reported a 95% confidence interval of 28% to 40% for the reduction, and said the agent used fewer tokens on 88% of questions. Its cache-aware API cost estimate fell from 7.3 cents per question with grep to 4.4 cents with delegation.
The savings came with a time penalty. Average time per question rose from 13 seconds with grep to 21.6 seconds with ColGrep-Agent, and the agent’s own call took a median of seven seconds on LightOn’s GPU server. The agent answered 80.8% of questions correctly, compared with 83.3% for grep; LightOn says that difference was not statistically significant, but its confidence interval does not rule out a meaningful accuracy decline.
A small model handles a narrow search job
ColGrep-Agent searches by meaning as well as exact terms. That matters when a developer describes a feature differently from the names used in the code: a question about “custom attributes,” for example, can lead to code that calls them “fields.” The model can repeat searches and inspect likely files, then return locations in a path-and-line-range format.
LightOn says the agent starts from MiniCPM5-2B and was trained with supervised fine-tuning followed by reinforcement learning. Its search loop has three tools: ColGrep for semantic and filtered searches, a read-only terminal for inspecting files, and a final submission tool. The agent has a 10-turn limit.
The standard install downloads a quantized model file of about 2.5 GB. LightOn provides installation routes for macOS, Linux and Windows; its model card says the local setup does not require Python, an external inference server or an API key. The repository needs an initial index, which the tool updates as files change.
The result is repository localization, not a fix
LightOn’s headline comparison measures code-search questions, not whether a coding assistant can complete a software issue from start to finish. Correctly identifying files and lines does not establish that an agent will make the right edit, pass the project’s tests or finish a task faster. The test also covers 52 questions, so the reported averages do not settle how the tool performs across other repository structures or search habits.
For developers, the practical tradeoff is clear in LightOn’s own figures: delegation lowered billed API tokens and estimated API cost, while adding about nine seconds to average question time and producing a slightly lower observed correct-answer rate. The release therefore makes its strongest case for workloads where repeated repository exploration is expensive enough to justify slower searches—and where the main coding agent can verify the returned locations before changing the code.