Can Test-Time Memory Evolution Outperform Test-Time Training for Scientific Discovery?

AI summary

Overview

• Research area: Test-time adaptation for large language models applied to automated scientific discovery (LLM-driven program search over mathematical construction problems).
• Technical level: Intermediate — readers should know roughly what LLM inference, reinforcement learning fine-tuning, and search archives are, but the paper's core idea is explained in plain terms.
• Scope: The paper proposes Test-Time Memory Evolution (TTME), an external-memory alternative to test-time training (TTT), and compares the two under identical discovery frameworks across five mathematical tasks and three models.

What This Paper Is About

Recent frameworks such as TTT-Discover and ThetaEvolve let open-weight LLMs match or beat frontier-model systems on scientific discovery tasks by updating the model's weights online (via test-time reinforcement learning) as the search proceeds. That weight updating is expensive in compute and time, which makes it hard to scale to larger models or harder problems. The authors ask whether continually evolving an external memory, while keeping the model weights frozen, can produce equal or better discovery results at much lower cost.

Key Contributions

1. A direct head-to-head test of parametric vs. non-parametric adaptation. The authors keep each discovery framework's parent-selection rules, archive rules, and task verifier intact, and replace only the TTT component with their memory method, isolating the effect of the adaptation mechanism.
2. TTME, a plug-and-play self-evolving memory system built from two components: Guideline Memory, which distills execution and evaluation feedback into reusable operational guidelines, and Strategy Memory, which extracts algorithmic ideas from score-improving modifications and consolidates them into reusable "strategy families."
3. Strategy-Guided Search, a procedure that selects relevant strategies for a given parent, adapts them into parent-specific modification instructions, and mixes strategy-guided candidates with directly generated ones (controlled by a fraction α, set to 1/2 in experiments).
4. An empirical evaluation across two frameworks, three models, and five tasks, reporting both discovery scores and measured runtime/FLOP costs, plus ablations and an analysis of search behavior.

Main Findings

• TTME beats TTT in every reported setting. Under the TTT-Discover framework with Qwen3-8B, TTME reaches 0.380927 on Erdős (lower is better), 1.504329 on AC1 (lower is better), and 0.949248 on AC2 (higher is better), versus TTT's 0.380932, 1.50525, and 0.9472.
• The gains hold for a much larger model. With gpt-oss-120b under TTT-Discover, TTME scores 0.380862 on Erdős, 1.502745 on AC1, and 0.960784 on AC2, versus TTT's 0.380876, 1.50287, and 0.9591.
• The gains transfer to a second framework. Under ThetaEvolve with Distill-Qwen3-8B, TTME reaches 0.948148 on AC2, 1.464224 on AC3 (lower is better), and 0.579938 on Hadamard (higher is better), versus TTT's 0.9469, 1.4930, and 0.5764.
• Frozen-weight search alone is not enough. The "w/o TTT" baseline (fixed weights, no memory) is generally worse than TTME on all tasks, indicating the memory itself carries the benefit rather than merely avoiding weight updates.
• Large efficiency savings. Under TTT-Discover with Qwen3-8B, TTME cuts runtime excluding evaluation by 47.7–58.4% and FLOPs by 82.8–84.6%, even though TTT-Discover updates only LoRA parameters. Per task: AC1 11.76 h → 5.36 h (54.5%) and 29.47 → 4.53 EFLOPs (84.6%); AC2 12.73 h → 6.66 h (47.7%) and 33.09 → 5.69 EFLOPs (82.8%); Erdős 11.67 h → 4.85 h (58.4%) and 26.21 → 4.15 EFLOPs (84.2%).
• Faster progress within the budget. TTME surpasses the final TTT scores within 25 search steps across all three tasks and both models under TTT-Discover, and also surpasses TTT on the three tasks under ThetaEvolve.
• Adding TTT on top of TTME does not help. In a four-way ablation with Qwen3-8B, TTME alone is best on AC2 (0.949248) and Erdős (0.380927); the combined TTT+TTME setting degrades to 0.944141 and 0.380944.
• Both memories matter. With gpt-oss-120b, removing Guideline Memory gives 1.505644 / 0.958506 / 0.380865, removing Strategy Memory gives 1.505981 / 0.956493 / 0.380864, and removing both gives 1.505966 / 0.953682 / 0.380946 — all worse than full TTME's 1.502745 / 0.960784 / 0.380862.
• Better idea recombination. TTME selects candidates that combine a parent's idea with an idea from outside its ancestry more often than TTT: 10.2% vs 8.2% on AC1 and 24.7% vs 18.6% on AC2. Counting only combinations that improve on their parent by more than 10⁻⁵, TTME still leads: 5.4% vs 4.8% on AC1 and 17.1% vs 13.3% on AC2.
• Validity is not the whole story. Both TTT and TTME raise mean program validity above the no-adaptation baseline; on AC1 and Erdős the two have similar validity while TTME achieves better best-found scores, suggesting TTME produces more effective ideas rather than merely more runnable code.

Methodology in Plain English

The setting is a loop: an LLM writes a candidate program implementing a mathematical construction, a verifier scores it, and the best candidates are reused as parents for the next round. The question is how the system should learn from the failures and successes it accumulates.

The TTT approach trains the model online on that feedback, so the experience ends up in the weights. The authors' alternative leaves the weights alone and instead writes the experience into two external stores:

• Guideline Memory collects practical, implementation-level lessons (for example, a constraint that caused a crash or a choice that improved a score). Every K steps, the model summarizes recent outcomes, proposes new guidelines, and merges them with existing ones — incrementing a recurrence count for matches, dropping stale entries, and enforcing a capacity limit with recency as a tiebreaker.
• Strategy Memory focuses on ideas, not mechanics. The system picks out cases where a modification produced a valid program with a higher score than its parent, has the model extract "experience notes" describing the idea and how it was applied, then groups notes that share an underlying idea into a persistent strategy family.

At generation time, a selector chooses up to N strategies judged promising for a specific parent (preferring strategies not already used by earlier parents in the same step), and an adaptor turns each into a concrete modification instruction for that parent. If nothing suitable is found, the model proposes a new idea instead. Half the candidate budget per parent goes to direct generation conditioned only on the parent and the guideline memory; the other half is split across the strategy-guided instructions.

The evaluation is deliberately conservative: the authors reuse each framework's original configuration, keep the same search budget and verifier, and take the TTT numbers from the original TTT-Discover and ThetaEvolve papers. Tasks are five mathematical construction problems — Erdős's minimum overlap problem, three autocorrelation inequalities (AC1, AC2, AC3), and the maximal determinant (Hadamard) problem on a 29×29 ±1 matrix. Runtime and FLOP measurements include candidate generation, all memory-related LLM calls, and RL training where applicable.

Why This Matters

The paper reframes a design choice that is becoming central to LLM-based discovery systems: whether experience should be internalized in weights or externalized in memory. If external memory suffices, discovery pipelines become dramatically cheaper to run and easier to swap between models, since no training infrastructure or weight checkpoints are needed mid-run. The measured 82.8–84.6% FLOP reduction and 47.7–58.4% runtime reduction are the kind of savings that determine whether a search campaign is affordable at all.

Real-world applications:

• Mathematical optimization and constant-improvement searches, where the goal is tightening a bound or finding a better explicit construction, as with the autocorrelation constants and Erdős's minimum overlap problem studied here.
• Evolutionary program synthesis and automated code improvement, since the guideline/strategy split maps directly onto "what makes code run" versus "what makes code better."
• Combinatorial design, such as the ±1 matrix search behind the Hadamard task, which connects to coding theory and experimental design.
• Cost-constrained research automation, where labs want agentic discovery loops running on open-weight models rather than paying for frontier-model inference or GPU training throughout a run.

Industry relevance: the result argues against reflexively reaching for online RL whenever an agent needs to adapt. For teams building long-running agentic or scientific-computing products, a memory layer that leaves weights untouched is simpler to deploy, easier to audit, and far cheaper to serve.

Future Directions

• Scaling to harder and non-mathematical domains. The evaluation is confined to five mathematical construction tasks with executable verifiers; whether memory evolution holds up in domains with noisier or delayed feedback (biology, materials, theorem proving) is untested here.
• Scaling to larger models and longer budgets. The authors motivate the work by TTT's difficulty in scaling, and test models spanning 8B and 120B parameters, but the cost profile at much larger scales and over much longer searches remains open.
• Better memory maintenance. The current scheme uses a fixed update interval (K = 2), a single modification instruction per parent (N = 1), and capacity limits with recurrence-based eviction; retrieval quality, forgetting policies, and cross-run memory reuse are natural next targets.
• Understanding when memory and weight updates are complementary. In the reported ablation, combining TTT with TTME hurt performance on both tested tasks, which raises the question of whether any regime exists where the two adaptation mechanisms reinforce each other.

Target Audience

Researchers and engineers working on test-time adaptation, LLM agents for scientific discovery, and evolutionary program search — particularly those who already know frameworks like FunSearch, AlphaEvolve, TTT-Discover, or ThetaEvolve and want to understand whether online weight updates are actually necessary. It is also useful for practitioners who need to justify the compute cost of an agentic discovery pipeline, since the paper reports runtime and FLOPs alongside task scores rather than scores alone.

Authors’ abstract

Recent \emph{test-time training} (TTT) frameworks have enabled open-weight large language models (LLMs) to outperform inference-only frameworks powered by frontier models on scientific discovery tasks. However, TTT introduces additional computational cost for online weight updates during the discovery process, making computational efficiency an important consideration when scaling to larger models. This motivates us to explore an efficient \emph{test-time adaptation} method for scientific discovery. We introduce \emph{Test-Time Memory Evolution} (TTME), a plug-and-play test-time adaptation method, as an alternative to TTT. TTME continually updates the external memory by distilling the experience to guide subsequent discovery without updating model weights. To have a clean comparison, we evaluate TTME and TTT under the same discovery framework, replacing only the test-time adaptation method. Our evaluation shows that TTME outperforms TTT on all five mathematical tasks, while reducing computational cost by $82.8$-$84.6\%$ in FLOPs and $47.7$-$58.4\%$ in runtime. These results establish TTME as an effective and computationally efficient alternative to TTT in the evaluated discovery settings.