AutoNovo: Agentic Program Search Discovers Transferable Nucleic Acid Design Algorithms

AI summary

Overview

Research area: Machine learning for biological sequence design, specifically agentic AI (LLM-driven program search) applied to nucleic acid optimization, evaluated on the NucleoBench benchmark.

Technical level: Intermediate. The paper assumes familiarity with gradient-based sequence optimization, surrogate/oracle models, and the idea of LLM code-search loops, but its framing is accessible to anyone working on AI for biology.

Scope (one sentence): The paper asks whether an LLM-based evolutionary code-search system (ERA) can discover nucleic acid designer programs that beat human-written baselines and, importantly, transfer to predictive models the search never saw.

What This Paper Is About

Designing DNA in silico means optimizing a sequence against a predictive model (an "oracle") that estimates something like chromatin accessibility. The problem is that sequences optimized this way often exploit quirks of that particular oracle, so they score well in the model but may not work in real biology. The authors use agentic program search to automatically write, run, score, and rewrite designer algorithms, testing whether the discovered algorithms generalize to independent oracles that were hidden during the search.

Key Contributions

1. An agentic search pipeline ("AutoNovo") for nucleic acid designers. The authors wrap the ERA (Empirical Research Assistant) system around a fixed Designer interface — __init__ (stores model_fn and a length-preserving ACGT start sequence), run(allowed_runtime_secs), and get_samples — creating two nested optimization loops: ERA searches over programs, and each generated program searches over sequences.

2. A hidden-orthogonal-oracle objective for selecting generalizing programs. In four of five searches, ERA scores each candidate designer with f_eval = f_opt + λ·f_orth, where f_orth is an orthogonal oracle the designer cannot query (λ = 1 for BPNet ATAC or ChromBPNet; λ = 0.01 for Enformer, to account for score-scale differences).

3. A family of 16 discovered designers, not one universal winner. These come from two FastSeqProp code seeds (four each), one Ledidi seed (four), and four from a search against BPNet ATAC and ChromBPNet jointly. Best observed means are attained on 3/3 direct DNA tasks and all 6/6 cross-oracle transfer directions.

4. Practical harness guidance for agent-generated scientific code. The paper documents prompting strategies — banning try/except (AST-enforced, with worst-case scoring), seeding with descriptions of existing algorithms, exposing the wall-clock budget, telling the agent gradients are optional and costly, and penalizing crashes and timeouts.

Main Findings

• Best observed means on all three direct tasks. AutoNovo designers reach −61.17 on BPNet ATAC, −74.88 on ChromBPNet, and −3360.36 on Enformer, versus the strongest baseline means of −40.88, −60.08, and −124.53 (all three from Ledidi). Lower is better; values are means ± standard deviation over N = 20 start sequences with a 30-minute CPU budget.

• Improvements are broad, not a single lucky candidate. 13 of 16 AutoNovo designers beat the strongest baseline mean on BPNet ATAC, 7 of 16 on ChromBPNet, and 11 of the 15 completed candidates on Enformer.

• Best-scores differ by task. FSP-gradient memory STE wins direct BPNet ATAC, FSP-Best-Anchored STE wins direct ChromBPNet, and FSP-sparse restart STE wins direct Enformer.

• All six transfer directions have an AutoNovo winner. 15 of 16 designers beat the strongest baseline on BPNet ATAC → Enformer; 13 of 15 on Enformer → BPNet ATAC; 12 of 15 on Enformer → ChromBPNet. Transfer is weaker when ChromBPNet is the design oracle: only 4 of 16 beat the strongest baseline on both ChromBPNet → BPNet ATAC and ChromBPNet → Enformer.

• Transfer can cross a fully held-out oracle. FSP-Ledidi-PEX annealed STE, searched against BPNet ATAC and ChromBPNet with no hidden oracle, wins BPNet ATAC → Enformer even though Enformer was queried by neither the designer nor ERA.

• Direct score and transfer score are decoupled. FSP-Ledidi-PEX two-phase saliency has the weakest direct BPNet ATAC score among AutoNovo designers yet wins BPNet ATAC → ChromBPNet.

• One program transfers far beyond its search configuration. FSP-sparse restart STE was searched with ChromBPNet as f_opt and BPNet ATAC as f_orth, yet is the best designer when Enformer is the design oracle and wins both Enformer-origin transfer tasks.

• A shared algorithmic pattern emerged. The strongest programs combine continuous logit optimization with gradients, conversion to discrete sequences, explicit retention of the best discrete sequence found, and perturbation or restart when progress stalls. Eight of the nine winning table entries use a straight-through estimator (STE); every best observed mean comes from a FastSeqProp-seeded program.

• Seeds acted as loose priors, not constraints. Although the Ledidi description emphasizes an explicit L1 edit-distance penalty, none of the four Ledidi-seeded programs had one; they converged on the same ingredients as FastSeqProp-seeded programs (logits, Adam, STEs, best-sequence tracking, discrete mutations, recovery after stagnation).

• Memory efficiency beats batching on Enformer. FSP-diverse batch STE generates roughly thirty candidates during discrete search and runs out of memory when Enformer is the design oracle, while FSP-sparse restart STE follows a single STE trajectory, evaluates one discrete sequence at a time, checks candidates more frequently near the deadline, and is the strongest Enformer designer.

• Baseline rankings differ from NucleoBench. NucleoBench ran for 8 hours versus 30 minutes here (both on CPU), and hyperparameters were carried over from the 8-hour runs; Ledidi designs quickly relative to FastSeqProp, AdaBeam, and GrAdaBeam, which changes the ordering.

• Strong scores did not imply clean code. Several programs use sequence length as a proxy for oracle identity, contain unused controls, reuse stale gradients after abandoning gradient optimization, or carry comments that do not match executed conditions. None sets random seeds.

• The GrAdaBeam seed failed. ERA attempted to reimplement and improve GrAdaBeam but produced nothing competitive with the original, so GrAdaBeam appears only as a baseline.

• Oracle panel is correlated but not redundant. On the same N = 100 NucleoBench start sequences, DNA oracles correlate at Pearson r ≈ 0.73–0.81.

Methodology in Plain English

The setup has two layers. The outer layer is an LLM-driven code-search system, ERA, which uses tree search with an upper-confidence-bound rule to decide which candidate programs to rewrite and re-test. Each rewrite is executed end to end, a metric is read from the run, and search continues toward programs that improve it. The output is ordinary, inspectable Python.

The inner layer is the designer program itself, which edits a DNA sequence under a wall-clock budget. ERA was seeded with LLM-written implementations of four existing hand-crafted algorithms — GrAdaBeam, FastSeqProp, Ledidi, and Proximal Exploration — two implementations each, eight total. After an initial single-task round, the authors kept both FastSeqProp seeds, one Ledidi seed, and one PEX seed and ran five searches.

Each search used one of five oracle configurations: BPNet ATAC with ChromBPNet hidden, ChromBPNet with BPNet ATAC hidden, BPNet ATAC with Enformer hidden (λ = 0.01), ChromBPNet with Enformer hidden (λ = 0.01), or BPNet ATAC + ChromBPNet with nothing hidden. During search, candidate designers got 30 minutes per sequence on three NucleoBench start sequences, and the scored metric was the mean of f_opt + λ·f_orth across those three sequences. Crashes, try/except use, and overruns beyond a five-second grace period received a large penalty.

After search, the selected programs were frozen — no further code changes. Each frozen program was then instantiated with each of the three oracles as the design oracle, and every resulting sequence was scored by all three evaluation oracles. Same-oracle scores are "direct"; different-oracle scores are "transfer." Each AutoNovo designer also has one fully held-out task measuring algorithm-level generalization. Final evaluation used N = 20 start sequences with a 30-minute CPU budget.

The three oracles: BPNet ATAC (NucleoBench BPNet-lite, a dilated residual CNN trained on ENCODE K562 ATAC-seq, 3,000 bp window), ChromBPNet (fully convolutional dilated residual network trained on K562 ATAC-seq, ENCSR483RKN, with a frozen background bias model subtracted, 2,114 bp inputs), and Enformer (a long-range sequence-to-activity transformer over 196,608 bp, scored on K562 DNase tracks aggregated over the 128 bp bins overlapping the central BPNet window).

Why This Matters

Impact on research. The paper reframes nucleic acid design as a program-search problem rather than a hand-engineering problem, and it introduces cross-oracle transfer as the selection criterion during search rather than only as a post-hoc diagnostic. It also shows that the transfer objective can work without the designer ever querying the held-out model, hinting that selecting on the right oracle pair is itself a lever for generalization.

Real-world applications (as motivated by the paper):

• Therapeutic nucleic acid design, where the paper frames generalization as improving the odds that in-silico sequences behave as predicted and are therefore useful for therapy design.
• Faster, cheaper early-stage screening before committing to wet-lab synthesis and large experimental screens.
• Optimizing regulatory and expression properties such as translational efficiency, stability, and cell-type-specific expression — the scalar properties the paper describes oracles as predicting.
• Cross-model validation pipelines for any group that already uses orthogonal models to flag oracle-exploiting sequences.

Industry relevance. Biotech and pharma groups that use surrogate models for sequence optimization have a direct interest in designers that survive model swaps, since checking against independent oracles is already standard practice. For ML tooling vendors, the result is also a concrete demonstration that agentic code search can produce a reusable scientific algorithm — not just a one-off script — and the harness lessons (ban exception handlers, expose the time budget, make failures expensive) transfer to other automated code-discovery pipelines.

Future Directions

• Test AutoNovo on additional biological design problems beyond K562 chromatin accessibility, since the current evidence covers one phenotype with three related predictive models.
• Determine which algorithmic changes account for successful transfer, for example by ablating the STE, best-sequence anchoring, restart, and stall-detection components separately.
• Evaluate whether cross-oracle improvements predict wet-lab performance, since cross-oracle transfer does not establish biological validity.
• Discover a single designer that is superior in all contexts, rather than the current family of specialized programs — a goal the authors explicitly name for future work.
• Release and reuse the discovered designers, which the authors state they plan to do.

Target Audience

Most useful for ML researchers working on agentic code generation and program search, computational biologists and bioengineers doing in-silico sequence design, and practitioners who already use NucleoBench or similar oracle-based benchmarks and want to know which designer families hold up under model substitution. The harness and prompting observations in Section 6 are specifically valuable to engineers building LLM-driven code-optimization loops, and the code-quality caveats are important reading for anyone tempted to deploy agent-generated scientific code without inspection.

Authors’ abstract

In-silico optimization of nucleic acids relies on predictive oracles that imperfectly capture biological function, so sequences optimized against one model may exploit model-specific artifacts and fail to generalize to independent models. Recent studies have shown that not all designers generalize equally well to independent models. In this work, we use agentic program search to iteratively propose, execute, evaluate, and revise code for nucleic-acid designers. We describe the successful strategies used to prompt the search. AutoNovo designers achieved the best observed mean among the evaluated methods on 3/3 DNA-design tasks in terms of in-silico performance, and 6/6 transfer tasks in terms of independent models. The designers come from a family of discovered nucleic acid designers rather than a single universal best designer. We plan to release the resulting designers. These results are the first steps in creating nucleic acid designers that maximize cross-model generalization, and thus improving the odds that in-silico designed sequences reflect real biology.