The Pulse
SourceLearn Builds Persistent Models of Documents, Code and APIs
Researchers introduce SourceLearn, a method that lets AI agents refine a reusable model of a source as they study it and tackle related tasks. Across five benchmarks and three language-model backends, the system led in 13 of 15 test setting
AI.info Team · Written with AI ·
“We study source learning: developing reusable source-specific competence over a persistent authoritative source.”
Lucheng Fu, Kejing Xia, Yiyang Wang, Yiqiao Jin, Jinjin He, Xiyuan Yang, Haoxin Liu, Ye Yu, Haibo Jin, Yijia Xiao, Wenke Lee, B. Aditya Prakash and Haohan Wang
A research team introduced SourceLearn in a paper submitted to arXiv on October 1, 2026, proposing that AI agents should improve their understanding of a particular document set, code repository or API over repeated use—not just fetch relevant passages anew for each task. The system builds a persistent model of a source, then revises it as the agent studies the source and encounters tasks that expose gaps in its understanding.
The authors tested SourceLearn on five benchmarks using three language-model backends. It recorded the highest score in 13 of 15 benchmark-backend combinations, according to the paper. The results position the method as a complement to retrieval, not a replacement: agents still consult the original source for task-specific evidence.
SourceLearn treats repeated use as learning
Retrieval systems find information relevant to a prompt, while agent-memory systems preserve information from earlier interactions. The paper argues that neither necessarily builds a reusable understanding of how a particular source’s information connects, when a rule applies or how multiple pieces of evidence should be combined. SourceLearn instead keeps a revisable model of the source alongside direct access to it.
The method has two learning stages. Self-Directed Source Learning revisits parts the current model represents incompletely, looking for missing conditions, mechanisms or links between entities. Task-Guided Source Learning uses downstream tasks to identify both local gaps—such as a missing fact needed to answer incorrectly—and recurring needs in how the source should be organized.
Those signals guide what to inspect, but they do not become stored knowledge directly. The system rereads the relevant source material and reconstructs updates from it, leaving the original source as the factual authority. At answer time, an agent combines retrieved evidence with relevant portions of its accumulated source model.
Five benchmarks, three model backends
The evaluation covers document question-answering, code question-answering, API use and interactive tasks. Its benchmarks are MultiDoc2Dial, NarrativeQA, SWE-QA, APIBench and AppWorld; the three tested backends are GPT-5.6-Luna, gpt-oss-120b and DeepSeek-V4.1-Flash. The comparisons include Hybrid RAG, which retrieves source passages for each task, as well as RAPTOR and HippoRAG 2 for persistent source representations, and AWM for retaining task experience.
The largest reported margin comes on APIBench with GPT-5.6-Luna: SourceLearn scores 70.4% against 47.8% for Hybrid RAG, a 22.6-point difference. Across all five benchmarks with that backend, the paper reports SourceLearn ahead of Hybrid RAG by an average of 14.3 points. Its advantage was not universal: with gpt-oss-120b on AppWorld, SourceLearn scored 27.4%, below Hybrid RAG’s 30.4%.
To test whether the method merely benefits from receiving more context, the authors also gave retrieval-based comparison systems matching per-question context budgets on four sources. SourceLearn remained the top scorer on three and placed second on the fourth, Sphinx. The paper also reports that source learning changes what the model represents: across examined sources, the share of model units with explicit applicability conditions rose from about 30% in the initial model to 65%–82% after task-guided learning.
The source model still needs a source
The authors’ grounding audit shows why they keep direct source access in the design. In sampled units from three sources, direct contradictions were at or below 1%, but some task-guided units for document and code sources were only partially supported or unsupported. In the MultiDoc2Dial StudentAid sample, for example, 13.3% of task-guided model units were judged unsupported and 0.8% contradicted the cited evidence.
The results are evidence for a method tested under repeated tasks on persistent sources, not proof that it works equally well for every agent or source. The paper’s authors say their study focuses on stable, authoritative sources and identify evolving, noisy or conflicting material as an open limitation. For those settings, the original source remains the reference—and the learned model, a fallible aid to using it.