Low-N Protein Activity Optimization with FolDE

AI summary

Overview

Research area: Machine learning for protein engineering — specifically active learning-assisted directed evolution (ALDE) under low experimental budgets.

Technical level: Intermediate. The core ideas are accessible, but the paper assumes some familiarity with protein language models, directed evolution, and Bayesian optimization.

Scope: The paper introduces FolDE, an ALDE workflow that combines naturalness-based zero-shot selection, neural network activity prediction with ranking loss, naturalness warm-starting, ensembling, and constant-liar batch selection, and benchmarks it against three baselines on 20 ProteinGym targets (arXiv:2510.24053v1 [cs.LG], 28 Oct 2025).

What This Paper Is About

Proteins are usually improved by building and measuring many mutants, which is expensive, and many targets lack high-throughput screens — limiting researchers to dozens of mutants per campaign. Active learning for directed evolution (ALDE) reduces this cost by predicting which mutants are worth testing and iteratively updating the predictions, but existing ALDE methods tend to select the highest-predicted mutants each round, producing training data that is too homogeneous to build good models in later rounds. This paper builds FolDE to maximize success at the end of a campaign rather than just within a single round.

Key Contributions

1. FolDE, a complete low-N ALDE workflow with five components: zero-shot naturalness-based selection in round-1, a neural network top layer trained with ranking loss, naturalness warm-starting, ensemble predictions, and constant-liar batch selection.
2. Naturalness warm-starting, a procedure that pretrains the activity-prediction network to recapitulate protein language model naturalness scores for all possible single mutants before fine-tuning on measured activities, addressing the round-1/round-2 tension caused by biased first-round data.
3. A constant-liar batch selector for diversity-aware selection, parameterized by alpha, which applies pessimistic assumptions about selected mutants to downweight similar candidates in later selections. The authors note that to their knowledge no prior ALDE work employs batch-aware acquisition strategies.
4. An open-source release: the FolDE implementation and simulation software under a modified BSD license, incorporated into a deployable web application called Foldy.

Main Findings

• FolDE beats the EVOLVEpro-representative baseline. Across all 20 test set proteins, FolDE discovered a median 23% more top 10% mutants than the random forest baseline (p=0.005, one-sided Wilcoxon test, range 0.62×–2.32×) and was a median 55% more likely to find a top 1% mutant (p=0.037, range 0.25×–3.0×).
• FolDE also beats zero-shot naturalness selection. It discovered 6% more top 10% mutants (p=0.009, range 0.83×–13.6×) and had a 15% higher probability of finding a top 1% mutant (p=0.003). Zero-shot failed to find any top 1% mutants in 5 of 20 targets, whereas FolDE achieved nonzero probability for all targets.
• Single-mutation benchmark. FolDE increased top 10% mutants discovered by 13% over random forest (p=0.032, range 0.62×–2.14×), with improvements in 13 of 17 targets, and improved the probability of discovering at least one top 1% mutant within three rounds by 50%, with improvements in 11 of 17 targets.
• Multi-mutation benchmark. Relative to random forest, FolDE found a median 96% more top 10% mutants (range 1.7×–2.3×) and increased the median probability of finding a top 1% mutant by 2.25× (range 2.0×–2.5×). FolDE did not improve much on the zero-shot baseline here; both methods selected very high quality batches.
• Warm-starting fixes a prediction collapse. Without warm-start, models trained on naturalness-selected round-1 mutants collapsed in round-2 prediction quality (Spearman rho dropping from 0.48 to 0.04), while models trained with randomly selected mutants retained higher correlation (rho 0.27 in round-2). Warm-started models maintained rho approximately 0.48 across rounds 1–3. This translated to discovering 32% more top 10% mutants and a 23% higher probability of finding a top 1% mutant compared to FolDE without warm-start.
• Naturalness correlates with activity. Zero-shot naturalness-based round-1 selection yields 3.8× more top 10% mutants and a 3.6× higher chance of finding a top 1% mutant than random selection, with a Spearman rho of approximately 0.48, but the resulting mutants cluster in high-naturalness regions.
• Constant-liar improves diversity but had limited effect on headline metrics. In extended simulations, the number of new loci in round-2 batches increased from 9.5 and 3.8 (alpha=100) to 12.2 and 7.3 (alpha=1.0) on the single- and multi-mutation benchmarks respectively. As an illustration of homogeneous batches, in round-2 of the multi-mutation benchmark, a batch of 16 double mutants targeted only 4 out of 32 mutations at new loci.
• Alpha selection. Values between 3 and 6 slightly improved the probability of finding a top 1% mutant while slightly decreasing the number of top 10% hits; FolDE uses alpha=6 in round-2 only, reverting to pure exploitation (alpha=100) in round-3. On the test benchmarks, constant-liar showed minimal impact on the primary metrics.
• Ablation. The components contributing most to performance were zero-shot prediction in round-1, naturalness warm-start, and the use of ranking loss. Constant-liar, at the settings used in the simulation, did not have a large effect on top-line metrics. UCB had no effect on batch diversity.
• Ranking loss versus alternatives. A neural network top layer trained with ranking loss found slightly more top 10% mutants than random forest and many more than a neural network trained with regression loss.

Methodology in Plain English

The authors built FolDE around a simulation benchmark using pre-collected deep mutational scanning data from ProteinGym. They split the data into a 9-protein training set for development and a test set of 17 single-mutation plus 3 multi-mutation datasets. Each simulated campaign runs three rounds of 16 mutants, totaling 48 mutants, mimicking a low-throughput real campaign. They held out 50% of mutants per run to enable bootstrapping and provided no "initial labeled data."

The workflow runs as follows. In round-1, with no activity data, mutants are ranked by "naturalness" — the log-likelihood ratio of mutant to wild-type amino acids computed in a single pass through a protein language model — and the top 16 are selected. In later rounds, sequences are embedded using ESMC-300M (mean-pooled final hidden layer), an ensemble of five multilayer perceptrons maps those embeddings to activity predictions, and a batch is selected using constant-liar.

Training happens in two phases. First, the networks are warm-started by training on the naturalness scores of all possible single mutants, giving the model information about broadly tolerated substitutions that may be absent from the biased first-round picks. Second, they are fine-tuned on accumulated activity measurements using a Bradley-Terry ranking loss, which learns from pairwise comparisons and is invariant to the scale or units of the training data — this is what allows the two-phase procedure to work across differently-scaled data. Pairwise data is split 80-20 into train and validation sets using breadth-first search so the training set cannot trivially solve validation pairs.

Constant-liar builds batches iteratively: after picking the highest-scoring mutant, it imagines that mutant performed poorly (a pessimistic "lie" set to the minimum candidate label), propagates that assumption through the ensemble's covariance structure, and downweights similar mutants in subsequent picks. The alpha parameter, implemented as scaled observation noise, controls how confident the lie is, and therefore the exploration-exploitation balance.

Success is measured by cumulative top 10% mutants discovered and the probability of finding at least one top 1% mutant within three rounds — chosen because they reflect practical campaign goals better than correlation-based metrics like Spearman rho. Statistical comparisons use a one-sided Wilcoxon signed-rank test on log-transformed differences. Baselines are random selection, zero-shot naturalness selection, and random round-1 selection followed by ESM2-15b embeddings with a random forest top layer (representing EVOLVEpro).

Why This Matters

This work frames protein optimization as an iterative, data-limited problem where decisions in one round determine the data available later, and it shows that the standard practice of greedily selecting top-predicted mutants can actively degrade later-round models. The findings argue for evaluating activity predictors with iterative benchmarks rather than correlation metrics alone, and they demonstrate that low-N refinement of foundation model predictions through active learning can improve experimental outcomes. The authors suggest the same constraints may apply to other foundation models for regulatory elements, genomes, and whole-cell models.

Real-world applications cited or implied:

• Therapeutic development — the authors name resource-limited labs working on therapeutics as a main beneficiary.
• Sustainable biomanufacturing — listed among the beneficiary areas.
• Pandemic preparedness — listed among the beneficiary areas.
• General laboratory protein engineering — the open-source, web-application release lowers the barrier for labs without specialized machine learning expertise; the authors frame FolDE as moving protein optimization toward a routine effort that "any researcher can perform in a matter of months."

Industry relevance: the efficiency gains could let campaigns explore mutation spaces deeper than three mutations under the same experimental budget, and the tools are freely available under a modified BSD license, meaning commercial and academic labs can adopt the workflow rather than building one from scratch. On ethics, the authors used the Responsible AI x Biodesign framework, concluded that FolDE is an incremental efficiency improvement rather than a categorically new capability, excluded viral proteins from benchmark development, and do not distribute protein language models or weights with their code.

Future Directions

• Testing whether more theoretically motivated acquisition functions help. The authors state that future work should explore whether methods like parallel knowledge gradient lead to improved performance in the search for high-activity mutants.
• Establishing the value of constant-liar in real campaigns. Constant-liar's diversity gains had minimal effect on the simulated primary metrics, and the authors suggest these metrics may underestimate its benefits in real campaigns exploring exponentially larger sequence spaces.
• Scaling beyond three rounds and three mutations. The paper argues that efficiency gains enable deeper exploration of mutation space so researchers can engineer more radical functional changes with the same budget, but comprehensive datasets at those depths (six or more mutations, and up to 29 in some campaigns) remain unavailable.
• Extending the iterative lens to other foundation models. The authors propose that the same constraints and opportunities may apply to foundation models for regulatory elements, genomes, and whole-cell models.

Target Audience

This paper is most useful to protein engineers and metabolic engineers running low-throughput directed evolution campaigns, to machine learning researchers working on active learning, Bayesian optimization, or scientific foundation models, and to tool builders who want to implement or extend an open-source ALDE workflow. Labs without specialized machine learning expertise that want to adopt an off-the-shelf optimization pipeline are also a stated target, given the web-application release.

Authors’ abstract

Proteins are traditionally optimized through the costly construction and measurement of many mutants. Active Learning-assisted Directed Evolution (ALDE) alleviates that cost by predicting the best improvements and iteratively testing mutants to inform predictions. However, existing ALDE methods face a critical limitation: selecting the highest-predicted mutants in each round yields homogeneous training data insufficient for accurate prediction models in subsequent rounds. Here we present FolDE, an ALDE method designed to maximize end-of-campaign success. In simulations across 20 protein targets, FolDE discovers 23% more top 10% mutants than the best baseline ALDE method (p=0.005) and is 55% more likely to find top 1% mutants. FolDE achieves this primarily through naturalness-based warm-starting, which augments limited activity measurements with protein language model outputs to improve activity prediction. We also introduce a constant-liar batch selector, which improves batch diversity; this is important in multi-mutation campaigns but had limited effect in our benchmarks. The complete workflow is freely available as open-source software, making efficient protein optimization accessible to any laboratory.