KnowEvo: Knowledge Evolution for Protein Optimization

AI summary

Overview

Research area: Machine learning for scientific discovery — specifically, LLM-driven protein engineering and the design of self-improving, externalized knowledge representations.

Technical level: Advanced. The paper assumes familiarity with Bayesian posterior sampling (Beta–Bernoulli / Thompson sampling), LLM agent architectures, and protein fitness-landscape optimization; it also uses domain-specific scoring backends (Rosetta REF2015, SaProt).

Scope: The paper introduces KnowEvo, a framework that treats externalized, executable "expertise blocks" — rather than model weights or prompts — as the object being optimized, and validates it on two protein optimization tasks (stability and solubility).

What This Paper Is About

Large language models can propose scientific hypotheses and protein mutations, but they lack a structured way to accumulate, inspect, and refine reusable expertise across repeated experiments — knowledge typically ends up buried in model weights or scattered across one-off solution traces. KnowEvo addresses this by representing scientific knowledge as modular, executable strategy programs organized into mutation trees, then iteratively allocating evaluation budget across those programs and having a sandboxed code agent revise them based on experimental evidence. The goal is to make the knowledge itself the optimization target, so that expertise can be inspected, attributed to individual components, and progressively improved.

Key Contributions

1. Reframing scientific discovery as knowledge optimization. The authors reformulate protein optimization as iterative refinement of an externalized, persistent knowledge base rather than of model parameters, decoupling domain adaptation from expensive finetuning.

2. Executable expertise blocks organized as strategy mutation trees. Each piece of domain knowledge is concretized as a deterministic program with explicit inputs, parameters, and mutation outputs; validated revisions are appended as child nodes, preserving the revision lineage of each design mechanism.

3. A two-phase evolution loop. A forward phase uses a Thompson-style posterior allocation (Beta–Bernoulli with a softmax temperature) to distribute evaluation budget across strategy nodes, while a backward phase uses a code agent in an isolated revision sandbox to translate rollout evidence into localized program edits.

4. Empirical validation on protein stability and solubility. KnowEvo outperforms root seed blocks, a monolithic program baseline, GEPA, AdaEvolve, and an agentic variant of AdaEvolve across pass@1/3/5, MRR, MAP, and NDCG@5, with supporting ablations, initialization-sensitivity tests, a cross-predictor check, and scaling analyses.

Main Findings

• Stability (Rosetta REF2015): KnowEvo raises pass@1 from 42.4% (root seed blocks) to 50.5% and pass@3 from 74.2% to 82.8%; MRR goes from 59.1 to 66.4 and NDCG@5 from 65.2 to 71.2.

• Solubility (SaProt): Gains are larger — pass@1 rises from 31.6% to 51.5%, pass@3 from 57.7% to 83.2%, MRR from 47.2 to 67.8, and NDCG@5 from 57.5 to 72.8.

• Beats LLM-guided evolution baselines: KnowEvo is best on all six metrics on both tasks. Against AdaEvolve, stability MRR improves from 60.5 to 66.4; against Agentic AdaEvolve (same sandboxed code agent), KnowEvo improves MRR by 6.2 points on stability and 9.8 points on solubility.

• Every ablation hurts performance (solubility/SaProt): uniform random node allocation drops pass@1 to 51.0 and MRR to 65.4; single-step LLM revision (no sandbox) drops pass@3 to 80.1; chain-only revision (discarding ancestors) collapses badly to pass@1 43.4 and pass@3 54.1, versus 51.5 / 83.2 for the full method.

• Robust to initialization: Starting from cold-start random roots still yields pass@1 46.4 and pass@5 93.9; warm-start reaches pass@1 54.6, and LLM-generated initialization reaches pass@3 85.7 and NDCG@5 74.5, versus 83.2 and 72.8 for human-written initialization.

• Gains transfer to an independent predictor: Re-scoring with ThermoMPNN (never used for evolution) gives KnowEvo pass@1 15.7 vs 9.1 for initial strategies, and MRR 23.7 vs 20.5, though these are point estimates only.

• Scaling with revision budget: Averaged over six independent revision trajectories, going from 10 to 100 accepted revisions raises pass@3 from 59.5% to 74.2% and lowers average first-hit rank from 2.58 to 2.01; the authors note these best-so-far curves are non-decreasing by construction.

• Scaling with agentic work and backbone choice: Higher sandbox work budgets (measured by cumulative agent turns) yield larger endpoint gains under the same revision-count budget, and a tested DeepSeek revision backbone outperforms MiniMax-M2.7 on all reported endpoint metrics in that diagnostic.

• Mechanistic refinement is traceable: In the solubility multi-site lineage, strategies progress from combining hydrophobic/charge/aromatic edits to conditioning substitutions on global composition, local context, and sequence position — e.g., the node 9 → node 14 revision coordinates local sequence context with protein-wide composition and adds N-terminal protection.

Methodology in Plain English

Instead of tuning a model's weights or rewriting one giant prompt, the researchers wrote the domain knowledge down as small, runnable programs — each one a specific "design mechanism" (for example, a rule about when to make a charge-changing mutation). These programs are grouped into families, and each family is a tree: the root is the original human-written rule, and every child is a revised version that was accepted after testing.

Training alternates between two loops:

• Forward (test the rules): For each protein, candidate rules are sampled and each one proposes a mutation. The mutation is scored by the task backend — Rosetta energy for stability, a SaProt classifier for solubility — and counts as a success if it improves on the parent. Which rule gets tested next is decided by a Bayesian bandit: each rule keeps a running success/failure count, the framework samples a plausible success rate from that distribution, and a temperature-controlled softmax converts those samples into selection probabilities. Untested rules start with a uniform prior, and the fixed positive temperature guarantees every eligible rule retains some chance of being picked.

• Backward (fix the rules): Once a rule has accumulated enough test trajectories, a code agent takes over inside an isolated "sandbox." The sandbox holds the program, the rollout evidence, runtime traces, task schemas, analysis scripts, and validation tests as persistent files, so the agent can inspect them selectively with tool calls rather than receiving one compressed prompt. It analyses, edits the code, runs validations, and repairs failures. Only revisions that pass the runtime contract and validation are accepted and appended to the tree.

Final selection picks one node per family by maximizing a lower confidence bound (the 5th percentile of its Beta posterior), and candidates are ranked by those LCB scores. Experiments used 5,000 training proteins and a 200-protein held-out split from BRENDA (198 usable for stability, 196 for solubility), with a 5,000-rollout budget for search-based methods and MiniMax-M2.7 as the code agent.

Why This Matters

Impact on research. The paper pushes against the default assumption that improving an LLM on a scientific task means updating its weights or tuning its prompts. By making knowledge an explicit, modular, executable artifact, it makes scientific heuristics individually inspectable, individually revisable, and individually attributable — which addresses a real credit-assignment problem when many mechanisms jointly determine an outcome. The finding that cold-start and LLM-generated initializations are competitive is particularly useful, since it suggests the framework does not depend on scarce hand-written expert rules.

Real-world applications:

• Therapeutic protein and antibody engineering — improving stability or developability while keeping the ranked-candidate output usable by wet-lab teams.
• Industrial enzyme development — raising thermal stability or soluble expression for biocatalysis and manufacturing, where Rosetta-style energy scoring is already standard.
• Automated lab-in-the-loop pipelines — the sandboxed revision loop maps naturally onto design-build-test-learn cycles where each round of assay data should improve the next round's design rules.
• General scientific design tasks beyond proteins — the architecture is domain-agnostic: any setting with a programmatic evaluator and modular mechanisms (materials, small molecules, circuit design) could in principle adopt the same evolve-blocks pattern.

Industry relevance. The approach avoids finetuning costs and exposes an auditable knowledge base, which matters for regulated domains where "the model learned it" is not an acceptable explanation. Persistent lineages of accepted revisions give a reviewable record of why a design rule changed — and the paper's token-usage analysis (Appendix H) signals awareness that agentic revision has real cost implications at deployment.

Future Directions

1. Multi-objective and multi-step optimization. All experiments used single-step rollouts (H = 1), so the framework's behavior under long trajectories with delayed, compositional rewards — where credit assignment is genuinely entangled — remains untested.

2. Principled budget allocation. The authors explicitly decline to claim a stationary-bandit regret guarantee for their growing-tree, softmax-variant Thompson sampling. Deriving guarantees for this adaptive, expanding candidate set is an open theoretical problem, and the ablation shows allocation policy already matters.

3. Better attribution when mechanisms interact. The paper's core premise is that knowledge is modular, but the evaluation section describes credit assignment across jointly-acting components as fundamentally hard; a method for isolating the marginal contribution of interacting blocks would strengthen the approach.

4. Larger-scale and alternative revision backbones. The DeepSeek-vs-MiniMax diagnostic suggests the code agent itself is a first-class lever. Systematically characterizing which agent capabilities drive revision quality, and how far gains scale with revisions and work budget, is a natural next study.

Target Audience

Researchers and engineers working at the intersection of LLM agents and scientific discovery — particularly those in protein engineering, computational biology, and automated experimental design. It is also relevant to practitioners of program/prompt evolution and agentic self-improvement who want a concrete architecture for structured, attributive knowledge refinement rather than end-to-end prompt tuning. Readers need a working grasp of Bayesian bandits and LLM agent tooling; the protein-biology content is comparatively accessible since the tasks are framed as black-box score improvements.

Authors’ abstract

Large language models (LLMs) have shown promise in scientific discovery, yet systematically accumulating and refining reusable scientific knowledge remains challenging. Existing approaches primarily improve LLM-based problem solving through parameter optimization or feedback-driven refinement of prompts, programs, and solution trajectories, leaving acquired expertise either embedded in model weights or scattered across iterative problem-solving processes. As a…