Skip the Compressor: Coding Agents Fix More Issues When They Pick Whole Files

AI summary

Overview

Research area: Software engineering with large language models — specifically context construction for LLM-based coding agents that resolve real repository issues.

Technical level: Intermediate. The ideas are intuitive (show whole files vs. compress tokens), but the paper assumes familiarity with retrieval, prompt compression, token budgets, and test-graded benchmarks like SWE-bench/SWE-rebench.

Scope: A controlled, budget-matched, test-graded comparison of three ways to fill a coding agent's context — token pruning, chunk retrieval, and exact whole-file selection — on 300 SWE-rebench issues with two model families.

What This Paper Is About

A coding agent must fix issues in repositories far larger than its context window, so something has to decide what code the model actually sees. Two common answers are retrievers (which rank code by similarity to the issue) and prompt compressors (which delete tokens to fit a budget), but a patch requires two things at once: the files that must change, and their text exactly as written, since the model has to reproduce the lines it edits. The paper asks a practical question — at a fixed token budget, which way of compressing the repository lets the model resolve the most issues? — and argues the budget is best spent choosing a few whole files rather than pruning tokens or retrieving chunks.

Key Contributions

1. CQ2-CARE, a simple exact-file selector: it asks the model under test twice, independently, which files must change, keeps the files both answers name, adds short grep-derived evidence spans for definitions, callers, or requested strings, and shows the chosen files whole and verbatim — with no compression model in the loop.
2. A budget-matched, test-graded comparison of CQ2-CARE against three prompt compressors (LLMLingua-2, LongLLMLingua, LongCodeZip), three retrievers (chunk BM25, dense retrieval with CodeRankEmbed, and SWE-bench's file-level BM25), and two methods trained for issue resolution (SWE-Pruner, SweRank), on 300 issues and two model families.
3. A decomposition of the losses into two measurable quantities — gold-file recall (the share of files to edit that appear in the context) and the share of verbatim lines — plus controlled tests that change one factor at a time.
4. A practical rule for agent builders: choose the candidate files well, then cut whole lines if anything must be cut; token-level deletion costs roughly ten points even when the files are held fixed.

Main Findings

• Exact files win at equal budget: CQ2-CARE resolves 43.7% of the 300 issues with GPT-5.6 and 46.7% with Claude Opus 5, which is 6.3 points more than the best of three compressors with either model.
• It also beats retrieval by a wide margin: 9.3 points (GPT-5.6) and 12.7 points (Claude Opus 5) over the best of three retrievers.
• It edges out trained methods: it leads SweRank by 0.7 and 1.7 points and SWE-Pruner by 2.0 and 7.3 points; the SWE-Pruner gap is significant only with Claude.
• Whole files alone are not enough: SWE-bench's file-level BM25 shows whole files chosen by similarity to the issue and still trails by 9.3 and 12.7 points, so which files are chosen matters.
• It approaches the oracle: the gold-file context resolves 49.3% with GPT-5.6 and 46.3% with Claude Opus 5, only 5.7 points above CQ2-CARE with GPT-5.6 and no more with Claude.
• Retrievers miss the files to edit: chunk BM25 and dense retrieval place a fifth or less of their context lines in the files the gold patch modifies (20.9% and 16.5%), against 51.7% for CQ2-CARE, and show none of those files on over a quarter of issues (26.2% and 27.5%).
• Compressors alter the text: LLMLingua-2 keeps only 17.5% of its lines verbatim with GPT-5.6 and 8.2% with Claude, and only 70.7% / 80.7% of its patches apply, against 98.0% / 83.3% for CQ2-CARE. Non-applying patches account for 28 of the 48 issues CQ2-CARE resolves and LLMLingua-2 does not.
• Alteration costs about ten points on its own: deleting 20% of the tokens of every line in CQ2-CARE's own contexts, files unchanged, drops patch applicability from 98.0% to 78.7% and resolution by 9.7 points (p<0.001), while a second patch sample of the intact contexts resolves exactly as many issues.
• Given the same files, a line-level compressor matches it: when LongCodeZip cuts CQ2-CARE's own candidate files instead of the larger single-draw context, it resolves 44.7% against CQ2-CARE's 43.7%; LLMLingua-2 comes within 3.0 points when its scores choose whole lines but trails by 7.3 when they choose tokens.
• Agreement is not the magic: agreement between two localization samples raises file-level F1 to 0.613 versus 0.430 for a single draw, but a single draw cut to the same number of files reaches 0.600 and resolves about as many issues — the benefit comes from naming few files well.
• The ordering holds under stress: across compression rates of 25%, 50% and 75%, within a ±2% budget tolerance, with search/replace edits instead of diffs, and with a second patch sample, no compressor or retriever reaches any exact-file context.
• It is cheaper: CQ2-CARE uses 0.68× the tokens of the uncompressed single draw with GPT-5.6 and 0.73× with Claude, at $0.151 per issue versus $0.199 for the single draw and $0.168–0.177 for compressor contexts in API calls alone; the compressors also spent a median 34–37 seconds of GPU time per issue (84–97 seconds on Claude's larger contexts, with LongCodeZip running out of memory on 4 issues at 48 GB).

Methodology in Plain English

The authors treat the repository as the agent's memory and the prompt as a compressed view of it. Every method they compare maps an issue plus the repository to a fixed token budget, after which one fixed model writes one patch that the benchmark's own tests grade.

The budget is set per issue by CQ2-CARE itself: they build its context first and use its realized token count as the budget every compressor and retriever must hit (compressors within ±2%). Retrievers pack ranked units verbatim; compressors shorten a larger single-draw context by deleting what a scoring model rates least useful; exact-file selection predicts the files to touch and shows them whole.

CQ2-CARE works in three steps. The model sees only a path inventory — paths ranked by shared words with the issue, capped at half of min(|P|, 32K) tokens, listed alphabetically — and returns ranked answers to "which files must change?". Files named by both of two independent answers are kept, minus test and documentation directories, with a fallback to a third answer if that intersection is empty. The model then requests at most four typed searches (definitions, callers, literal strings), and grep adds short spans from at most two new files each, up to eight files. The final context concatenates the selected code files in full (each capped at 60K tokens) and the evidence spans (each capped at 12K tokens), skipping any block that would push past 80K tokens.

To explain why methods win or lose, the authors measure each context offline on gold-file recall and the share of verbatim lines — metrics computed from the gold patch that no model ever sees — and then run controlled tests that hold one factor fixed while changing the other.

Data: SWE-rebench at a pinned revision with 860 Python tasks from 413 repositories. A salted hash drew 340 candidate issues, one per repository; 328 passed an eligibility check (gold patch resolves, empty patch does not), and the first 300 became the population, created between January 2025 and May 2026. A hash-chosen subset of 100 is used for the rate sweep and the edit-format control. Comparisons are paired by issue using one-sided exact McNemar tests, Holm-corrected within families, with 95% issue-bootstrap intervals; patches that fail to parse or apply count as unresolved.

Why This Matters

Impact on research. The paper reframes context management for coding agents as a selection problem rather than a compression problem, and shows that intermediate metrics can mislead: agreement between localization samples raises file-level F1 far above a single sample yet resolves about as many issues, and exit-code grading credited an empty patch on 4 of 6 issues. It argues context methods should be compared end to end, at equal budgets, with the benchmark's tests.

Real-world applications.

• IDE and coding assistants that must decide which files to paste into a limited context window before proposing an edit.
• Autonomous issue-fixing agents operating on large repositories where a full checkout cannot fit in any prompt.
• Code review and patching tools that rely on quoting exact spans or numbered line ranges, which requires text the model can match byte-for-byte.
• Cost-constrained deployment: a selector that needs no compression model removes GPU inference from the loop and lowers per-issue API spend.

Industry relevance. Any team shipping an LLM agent over a large codebase must choose between retrieval, pruning, and file selection. This paper gives a concrete default, a mechanism (agreement plus grep-based evidence), and a cost comparison, and it identifies a hard constraint: edit formats that replace a quoted span or a numbered line range break when the context is not verbatim.

Future Directions

• Full agent loops. The study uses one patch call per context. Agents that can re-open files may recover some losses, since a compressed view still tells them which file to read, but each re-read costs a turn and the tokens compression was meant to save, and a model may not notice that the text it sees is altered — measuring this trade-off is left to future work.
• A code-aware compressor starting from good candidates. The results do not show a selector beating such a method; given the same files, LongCodeZip matched CQ2-CARE. Testing this directly, and at what compute cost, remains open.
• Broader settings and benchmarks. The limitations note the study covers one patch call on Python issues from one benchmark, with most cells having a single patch sample (a second flips 8.7–18.9% of outcomes), so small gaps such as Claude's 6.3-point lead over LLMLingua-2 (p=0.027) are fragile.
• Validating the diagnostic estimator. The estimator built on gold-file recall and verbatim lines uses the gold patch, serves only as a diagnostic, and its coverage choice inside functions was made after seeing SweRank's result, so it needs a test on an unseen method.

Target Audience

Researchers and practitioners building LLM-based coding agents or context-management pipelines for large codebases; software engineering researchers who benchmark issue resolution; and engineers choosing between retrieval, prompt compression, and file selection under a fixed token and cost budget. Readers without background in retrieval or agent benchmarks will follow the intuition but may need the related-work section for context.

Authors’ abstract

A coding agent works on repositories that can be far larger than its context window can hold. Prompt compressors and retrievers offer solutions to this problem, but patching an issue needs two things at once: the files that must change, and their code exactly as written, since the model must reproduce the lines it edits. Retrievers rank code by its similarity to the issue rather than by whether it must change, and compressors delete text to fit the budget. We show that at an equal token budget, an agent resolves more issues when the budget goes to choosing which files to show verbatim, as our simple selector CQ2-CARE does, than when it goes to pruning tokens or retrieving chunks. We fit every context to the same per-issue budget on 300 recent SWE-rebench issues and grade each patch with the benchmark's tests. With GPT-5.6 and Claude Opus 5, CQ2-CARE resolves 43.7% and 46.7% of the issues, 6.3 points more than the best of three compressors with either model, and 9.3 and 12.7 points more than the best of three retrievers. CQ2-CARE even resolves more issues than two methods trained for issue resolution, SweRank (by 0.7 and 1.7 points) and SWE-Pruner (by 2.0 and 7.3). Our analysis shows the defects of the baseline methods: retrievers often miss the files to edit, and compressors keep them but alter their text, so fewer patches apply. Exact-file selection needs no compression model and shows every line as the repository has it, so the model can quote the code it edits.