The Pulse
Language model improves its own weights in preprint test
A 27B language model edited its own weights without a training run and improved on unseen problems, researchers report. The team also tested whether the model could repair a damaged copy.

An open 27B language model edited its own weights and became 6.6 percentage points more accurate on problems it had not seen, researchers report in a new preprint. The paper by Jiajun Fan and colleagues was submitted to arXiv on Nov. 16 and revised on Nov. 18.
The researchers call the approach self-rewriting. Rather than using another model to make the changes or running a conventional training process, they gave the model access to its own weights, graded practice answers and tools to run tests on itself.
How the edit worked
The model could take one action per turn: run code over its weights, test an edit or inspect its hidden states. The authors say the procedure used no gradients, optimizer or stronger model; the submitted edit was then evaluated on problems held back from the model.
In one experiment, the model examined differences in its hidden states and edited where it had measured a difference. The team tested the edited model on 490 held-out problems, with eight answers for each problem, using a paired evaluation fixed in advance.
In a sequence of eight sessions using a protocol that checked edits on reserved practice problems before submission, the accuracy of the model in use rose or stayed level, the paper reports. The edits from sessions two and eight were significantly better than the starting model on the held-out problems, while the model withdrew its edits in five sessions.
Each session began from the base weights, rather than starting with the weights submitted by the preceding session. A submitted edit replaced the model in use; a withdrawn edit left it unchanged.
Repairing a damaged copy
In a separate experiment, the researchers overwrote one block of a copy’s weights with noise and told the model that some values had been corrupted. The undamaged model edited the damaged copy without retraining, according to the paper.
The researchers say the model located the corrupted matrix without reading the undamaged weights, and wrote its own values rather than restoring the originals. The repair was developed using text math problems, then tested on image tasks it had not seen.
Across six unseen image benchmarks, with 100 problems in each, the repaired copy reached 83% of the base model’s accuracy, the authors report. On ScreenSpot-v2, they report that accuracy rose from 0% for the damaged copy to 89% after repair, compared with 93% for the base model.
Different results by size
The team also compared models from one family at 2B, 4B, 9B and 27B parameters, giving them the same task and tools. The 2B model made no edit, while the 4B and 9B models damaged their own performance, according to the paper. Only the 27B model improved itself and repaired its damaged copy in those tests.
The paper notes a difference in the session budgets: at the budget shared by all sizes, the 27B model reached more of the steps involved in rewriting itself, while other sessions on its own weights stated a shorter token budget that left time for more tests. The authors say each session behind the largest gains ran for a few hours on a single GPU, and that the resulting edit stays in the weights without adding computation to each subsequent answer.
Sources
- Original source · Preprint Language Models Can Rewrite Their Own Weights arXiv:2611.21885
Related stories
- IBM releases Granite 4.1 3B LLMON model in preview
- 35Gates offers dedicated cloud AI environments from ¥64,000 a month before tax
- Physicist says GPT-6 Astra helped develop Standard Model proposal
- Anthropic mathematician claims Claude helped prove Riemann hypothesis
- Anthropic says Claude solved two black-hole physics problems