PreciseCoder: Faithful and Generalizable Debugging Beyond Unit-Test Rewards

AI summary

Overview

Research area: AI for software engineering — specifically reinforcement learning (RL) post-training of large language models (LLMs) for automated program repair and debugging.

Technical level: Intermediate (accessible to readers with basic familiarity with LLM fine-tuning and reinforcement learning; the reward design itself is explained largely in plain terms).

Scope: The paper diagnoses why unit-test-only RL makes code models over-edit and hallucinate bugs, then proposes a semantically verified precision/recall reward ("PreciseCoder") that fixes both problems across 4B, 9B and 27B models while generalizing to repository-level repair and even transferring to frontier models through the reasoning trace alone.

What This Paper Is About

The standard recipe for post-training code models is reinforcement learning from unit tests: the model gets rewarded only for whether its repaired program passes. This paper audits what that reward actually teaches and finds that, although pass rates rise, the model learns to edit more lines than necessary and to invent bugs in programs that are already correct. The authors' goal is to make the reward "see" the edit itself — rewarding repairs that are both correct and minimal, in a way that stays stable during training and generalizes well beyond the training data.

Key Contributions

1. A diagnosis of unit-test-only RL. By tracking edit statistics over GRPO training rollouts and measuring correlations between candidate reward terms and correctness, the authors show that the unit-test reward raises pass rates while inflating edit size and suppressing the model's willingness to leave correct code untouched.

2. A new reward based on semantically verified localization. They introduce a fault-by-fault precision and recall computed by counterfactual test execution, combine a gated form of them into an F1 score, and mix this with the unit-test reward as r_λ = (1 − λ)·unit + λ·F1, optimized with GDPO-style per-term normalization (λ = 1/3).

3. Fidelity without a pass-rate trade-off. Across Qwen3.5-4B, Qwen3.5-9B and Qwen3.6-27B, PreciseCoder reduces average edit size to roughly 3 lines while keeping pass rate on par with unit-test-only GRPO, and it raises the no-edit rate on bug-free code from 31–37% to 47–52% despite never training on correct programs.

4. Generalization evidence, including transfer without weight sharing. Precision carries to multi-line and repository-level repair, SWE-Bench Verified, and multi-turn agentic editing; additionally, prefilling the thinking trace of a 9B PreciseCoder model into untrained frontier receivers such as GPT-5.6 Sol and Qwen3.8-Max raises their edit precision by 4.6–12.4 points.

Main Findings

• Unit-test RL buys pass rate with unnecessary edits. In a GRPO run on Qwen3.5-4B, the mean number of edited lines grew from 3.94 to 8.28 on the training set, and the model kept over-editing after the pass rate had plateaued.

• The same reward teaches bug hallucination. On a validation set of 200 bug-free programs, the no-operation rate fell from 30.8% at the start of training to 20.5% averaged over the last three evaluations, even though training never showed the model a correct program.

• Edit size is a poor training signal. The Pearson correlation between negative edit length and the pass indicator is only 0.13 on 4B and 0.14 on 9B. By contrast, the proposed precision term correlates with negative edit length at above 0.81, the recall term correlates with the pass indicator at above 0.71, and their F1 correlates with both (above 0.62 with the pass indicator, above 0.82 with negative edit length).

• Precision and pass rate are jointly achievable. On PDB Test with Qwen3.5-4B, PreciseCoder reaches 55.3% pass rate at 64.9% precision with 3.0 edited lines, versus 55.1% / 30.0% / 13.8 lines for unit-test-only GRPO and 53.3% / 58.6% / 3.7 lines for EA-GRPO. Baselines that penalize edit size lose pass rate: RECAP falls to 38.1% at 4B.

• Competitive at 27B. PreciseCoder on Qwen3.6-27B reaches 79.3% pass rate and 79.2% precision with 2.6 edited lines, exceeding DeepSeek-V4.1-Flash and Qwen3.8-Flash-Next on all measures. GPT-5.6 Sol has the higher pass rate (86.1%) but 61.5% precision and 4.1 lines.

• Restraint on correct code comes for free. Unit-test-only GRPO lowers the no-op rate (4B: 30.6% → 19.3%; 9B: 36.6% → 30.7%; 27B: 31.1% → 26.9%), while PreciseCoder raises it to 48.0%, 51.6% and 45.8% and reduces damage to 13.5%, 11.8% and 11.1% (base: 19.3%, 18.7%, 15.5%). The no-op rate and edit precision are near-linearly related (r = 0.87 across all models; 0.94 within 4B rows, 0.92 within 9B rows).

• Reward ablation confirms the design. Precision-only rewards reach 69.0–70.1% precision at 2.5–2.6 lines but pass 3–5% fewer tasks than the full reward; recall-only reaches the highest pass rate (57.2%) with the least precise repairs (52.2%, 3.8 lines).

• Generalization to harder repairs. On PDB-Wild (484 multi-line and repository-level tasks), precision rises by 20.3, 16.7 and 20.2 points at 4B, 9B and 27B, pass rate by 10–13 points at every size, and mean edit size falls from 37.8 to 10.8 (4B), 33.9 to 12.0 (9B) and 18.4 to 6.8 (27B).

• SWE-Bench Verified gains. With SWE-agent as harness, PreciseCoder-9B resolves 32.7% of issues (base 25.9%, GRPO 28.1%, EA-GRPO 27.1%); at 27B it resolves 70.8% (base 67.9%, GRPO 69.5%). Measured as excess changed lines over the gold patch on resolved instances, GRPO over-edits by 58% and EA-GRPO by 37%, while PreciseCoder-9B holds the excess at 29%. When the gold patch is applied before the agent starts, PreciseCoder-27B breaks 8.9% of repositories versus 11.0% for the base model and 13.1% for GRPO.

• Agentic multi-turn results. In a five-turn agent with view, str_replace_editor and submit, the single-turn-trained PreciseCoder-9B keeps 52.6% edit precision (GRPO: 42.9%) but trails by 2.9 points in pass rate. Trained with multi-turn agentic RL, PreciseCoder-9B-Agent reaches 67.4% precision at a 56.3% pass rate, the best of every 9B model on both.

• The thinking trace transfers across models. Prefilled into its own untrained base, the PreciseCoder-9B think block recovers 73% of the pass-rate gain and 60% of the precision gain from training, cutting the edit from 13.7 to 6.5 lines. Across five receivers, precision moves by +17.1 (Qwen3.5-9B), +2.6 (Qwen3.6-27B, control with an untrained 9B trace), +11.1 (Qwen3.6-27B continuing to think), +5.6 (Qwen3.8-Flash-Next), +5.5 (DeepSeek-V4.1-Flash), +12.4 (Qwen3.8-Max) and +4.6 (GPT-5.6 Sol), with pass rate moving between −1.5 and +4.3 points. The untrained 9B trace as a control costs the 27B receiver 7.4 pass-rate points.

• Linguistic analysis shows what changed. Under either reward the 9B model quotes code inline more often (170 and 168 spans per 1,000 words versus 111) and backtracks more ("wait": 12.3 and 15.2 versus 6.1 per 1,000 words). Only the precision reward adds judgement: more hedging (10.8 versus 6.5 per 1,000 words), less echoing of instructions (3.4 versus 6.5), and almost no arguing from the prompt that a bug must exist (0.04 versus 0.59). The share of tasks where the thinking names every buggy line verbatim rises from 36% to 67% at 4B and 43% to 71% at 9B; one standard deviation more quoting is worth +4.3 F1 and +3.2 precision points.

• Restraint arises from proposing and rejecting bugs. On the 1,512 correct programs of GT Debug, all models say the code "looks correct" at similar rates (8–11 times per 1,000 words), but PreciseCoder-9B proposes 27.6 bug hypotheses per 1,000 words versus 4.7 for its base and 10.8 for unit-test-only GRPO, and its no-op rate rises from 21% to 68% between the lowest and highest hypothesis quartiles.

• No loss out of distribution. On five benchmarks never trained on (BigCodeBench, LiveCodeBench, HumanEval+, CruxEval, DebugBench), PreciseCoder scores above its base model at both 4B and 9B; at 4B, GRPO loses 21.3 points on CruxEval.

Methodology in Plain English

The authors start by defining a debugging task as a buggy program, a specification, tests that the buggy program fails, and a reference repair. A policy model outputs a revised program, and the edit is the diff between the original and the revision.

Their key measurement is a fault-by-fault scoring scheme. Borrowing from prior work, each predicted edit block is matched to a reference fix block, and the model gets credit for a fault only if the program passes the tests when that single predicted block is substituted for the reference block. From this they compute recall (which faults were repaired) and precision (what fraction of edited lines were actually needed, using the smallest subset of the prediction that still fixes the fault, with a small tolerance for extra lines).

They then check which candidate reward terms correlate with what. A reward term is "stable" if it correlates with passing the tests, and "faithful" if it correlates with smaller edits among passing samples. Edit size fails the first test; the pass indicator fails the second; precision correlates strongly with small edits and recall with passing tests. Their F1 combination is the only term that correlates well with both. To avoid penalizing correct programs, recall is gated to 1 whenever the program passes, since a passing program has by definition fixed every injected fault.

The final reward mixes the pass indicator with this F1 score. A subtlety is that standard group-relative normalization applied to a sum behaves unevenly across groups, so they standardize each term separately within the group before mixing (GDPO), then rescale at the batch level. They derive that a passing correction retains a larger advantage than a failing one as long as λ < 2/(2 + √(2G)), which is λ < 0.37 at a group size of 6, and they use λ = 1/3.

Training uses 5,274 tasks from PDB-Single-Full (an unfiltered pool of 7,589 synthesized single-line-bug programs built on BigCodeBench and LiveCodeBench problems), testing on 1,205 held-out tasks from PDB-Single that share no base problem with training. Bug hallucination is measured on 1,512 ground-truth programs from BigCodeBench and LiveCodeBench problems outside those splits, using no-op rate and damage (test-failure) rate. Baselines are SFT with rejection sampling, DPO, unit-test-only GRPO, RECAP and EA-GRPO; frontier comparisons are GPT-5.6 Sol, DeepSeek-V4-Pro, DeepSeek-V4.1-Flash and Qwen3.8-Flash-Next.

For the weak-to-strong experiments, they take the full "think block" produced by a trained donor model and prefill it token for token into the assistant turn of an untrained receiver over a raw completions endpoint — no weight access, no shared tokenizer, no training — and the receiver either closes the block immediately or continues thinking from its end. For the linguistic analysis, they examine complete chains of thought and compare lexical categories, distinctive bigrams by weighted log-odds, and verbatim matches of the buggy lines.

Why This Matters

Impact on research. The paper reframes over-editing and bug hallucination as two symptoms of one cause — a reward that scores the program but not the edit — and shows they can be addressed jointly. It also provides evidence that a behavioral discipline (naming the buggy line, weighing whether an edit is worth keeping) can propagate from a small trained model to frontier models through text alone, which is a striking result for research on weak-to-strong transfer and on reasoning traces as a transferable artifact.

Real-world applications.

• Review cost in coding assistants: reviewers must inspect

Authors’ abstract

Reinforcement learning (RL) from unit tests is the common recipe for post-training code models, but a unit test only asks whether the final program passes. We analyze what this unit-test-only reward teaches a model during debugging and find that, while it raises pass rates, it also teaches the model to edit more and to hallucinate bugs in code that is already correct. We introduce PreciseCoder, a training algorithm that augments the unit-test reward with a signal for…