HypoEvolve: Genetic Algorithms Enable Multi-Agent LLMs to Discover Scientific Hypotheses

AI summary

Overview

• Research area: Automated scientific hypothesis discovery using multi-agent large language model systems combined with evolutionary/genetic search; evaluated on cancer drug repurposing.
• Technical level: Intermediate. The architecture is conceptually simple (a generational genetic algorithm wrapping three LLM roles), but the evaluation relies on domain-specific biological resources (DepMap CRISPR screens, Open Targets) and statistical machinery (Bradley-Terry ranking, paired Wilcoxon tests) that assume some familiarity.
• Scope: The paper proposes HypoEvolve, a framework that separates the scientific roles of LLM agents from the rules governing how their outputs are selected, combined, and retained, and tests whether that collaboration design — rather than the agents' underlying capabilities — improves hypothesis quality.

What This Paper Is About

Existing systems use multiple LLM agents to propose and critique scientific hypotheses, and separately use evolutionary search to improve generated artifacts, but it has been unclear how much of the resulting quality comes from the agents themselves versus the way their collective work is organized. HypoEvolve addresses this by keeping scientific roles, prompts, and evaluation criteria fixed while making the coordination rules explicit and testable: a fixed-size population of hypotheses is repeatedly scored by pairwise agent judgment and updated through LLM-driven crossover and mutation. The goal is to show that a specific style of collaboration — fitness-guided selection, semantic variation of scientific claims, and joint parent-offspring replacement — produces hypotheses better supported by independent biological evidence.

Key Contributions

1. A formulation of scientific collaboration as explicit population search. The paper separates "what agents reason about" (mechanistic arguments, assumptions, evidence, testability) from "how their outputs flow through the search" (parent selection, variation, replacement), turning collaboration design into an experimental variable rather than a fixed implementation detail.
2. A generational genetic algorithm instantiated with specialized LLM operators. Three roles — a generation agent for literature-grounded initialization, a pairwise scorer for comparative scientific judgment, and an evolution agent supplying crossover (combination, inspiration) and mutation (drug substitution, out-of-box) variants — are coordinated by a deterministic supervisor using (μ+λ) truncation with full parentage and operator provenance records.
3. A drug repurposing evaluation grounded in held-out external evidence. Hypothesis rationales are converted into a falsifiable target-level claim (the implicated targets are relevant to the specified cancer), then scored by two independent sources never used during search: DepMap CRISPR selectivity and Open Targets association, spanning 34 cancer types with a 27-type held-out protocol.
4. A controlled parent-selection ablation. Holding scientific operators, retrieval, evaluation, and hypothesis count fixed, the paper isolates the effect of fitness-guided versus uniform random parent selection on population quality.

Main Findings

• HypoEvolve scores highest on both external metrics. Mean DepMap selectivity reaches 0.171 versus 0.115 for Tree of Thoughts (the strongest baseline) and 0.039 for single-pass generation; Open Targets association reaches 0.426 versus 0.329 for Tree of Thoughts and 0.163 for single-pass. On the shared 26-type DepMap panel, the paired margin over single-pass generation is +0.133 with 19 wins and 7 losses.
• Gains transfer to held-out cancer types. Across the 27 held-out cancer types, with all protocol-development and interim-inspection types excluded, HypoEvolve exceeds single-pass generation by +0.280 on Open Targets and by +0.111 on DepMap (24 types with matched cell lines), using a frozen configuration.
• The advantage is largest on genetic evidence. Splitting Open Targets into evidence channels, the margin over single-pass generation is +0.247 for known drugs and clinical trials, +0.313 for literature, and +0.334 for genetic association — the strongest margin falls on the channel least directly documenting drug-disease pairs.
• Evolution improves drug-cancer matching, not just raw scores. Residual scores, which compare a proposed drug's performance in the matched cancer against its average across other cancers, rise across generations on both metrics (DepMap +0.0034 to +0.0612; Open Targets +0.0133 to +0.0658, peaking at generation 2), improving in 21 of 31 and 25 of 34 cancer types respectively.
• Most final answers are genuinely new, not carried over. Of 94 final hypotheses, 60 came from crossover (53 via inspiration) and 27 from mutation (23 via drug substitution); only 7 were unchanged parent copies produced by empty operator returns. Across all 1,692 offspring records, inspiration accounted for 27.6%, out-of-box mutation 24.8%, combination 22.9%, and drug substitution 17.1%.
• Fitness improves in every run. Population-mean fitness rose from 50.0 at initialization to 112.2 at generation 3, and best-member fitness from 81.2 to 125.7, increasing in all 94 runs under the agents' own comparative judgments.
• Fitness-guided parent selection raises the floor. Replacing fitness-based selection with uniform random selection, with everything else fixed, lowers the population minimum by 0.088 (DepMap) and 0.218 (Open Targets), the population mean by 0.075 and 0.128, and the final selected hypothesis by 0.090 and 0.156. Population maxima show no statistically detectable change.
• More search and more model calls are not automatically better. Raising the population to μ=10, extending to 5 generations, and changing operator rates produced mixed, non-significant shifts; disabling crossover, mutation, or both likewise produced no comparison surviving multiple-testing correction. HypoEvolve uses 206 model calls per run (167 of them pairwise comparisons) at roughly $0.56, fewer than the co-scientist scaffold (288 calls) and static reranking (227 calls), but far more than Tree of Thoughts (31 calls).
• Qualitative shifts in hypothesis content. In pancreatic adenocarcinoma, the leading hypothesis kept olaparib but narrowed a generic DNA-damage rationale into stratification by BRCA1, BRCA2, and PALB2; in melanoma the top candidate changed from trametinib to vemurafenib — illustrating both sharpening of conditions and outright substitution of the intervention.

Methodology in Plain English

The researchers treat hypothesis discovery as a small-scale evolutionary process. A "generation agent" first searches the literature on a research goal (a specific cancer type) and writes six structured hypotheses, each containing a title, summary, a hypothesis statement, and a rationale. Those six candidates compete in a round-robin of pairwise comparisons performed by a scoring agent, which judges which hypothesis is stronger based on specificity to the named cancer, evidence implicating the proposed drug target, and whether the hypothesis makes a concrete, falsifiable prediction. Those pairwise outcomes are converted into numerical fitness scores using a Bradley-Terry model, which produces a ranking from head-to-head results.

The next generation is created by repeatedly running a two-candidate tournament among the current population and letting the higher-fitness winner reproduce. With 60% probability, two parents are combined by an evolution agent — either merging compatible mechanisms ("combination") or borrowing ideas to construct a different explanation ("inspiration"). With 15% probability, or always when crossover is skipped, the result is mutated — either swapping in a different drug while keeping the mechanism ("drug substitution") or revisiting the hypothesis's underlying assumptions ("out-of-box"). Parents and offspring are then scored together in one pool, and only the top six survive. This means a new idea must outrank the ideas it was built from in order to be retained.

The key design choice is what is deliberately withheld from the search. DepMap and Open Targets scores, which depend on real CRISPR perturbation data and curated clinical/genetic associations, are computed only after the search finishes, on the single highest-fitness hypothesis. No method in the comparison is allowed to cherry-pick the best of its candidate pool against the external metric. All methods share the same base model, the same 61-drug vocabulary with annotated targets, the same retrieval setup, and the same single-hypothesis output format, so differences reflect search design rather than resources. DepMap scoring is made "selective" by subtracting each target's pan-cancer median dependency — a correction the authors found necessary because a constant answer (thalidomide) otherwise ranked first in 30 of 31 cancer types.

Why This Matters

Research impact: the paper makes an argument that is unusual in the agent literature — that the organization of a research team can be studied as an independent variable, not just the capability of its members. By fixing the agents and varying only the search rules, it provides evidence that coordination design contributes measurably to hypothesis quality, particularly for the average and weakest hypotheses in a population. It also introduces an evaluation pattern (generate free-form scientific explanations, then convert them into a falsifiable claim that external data can score, with that data withheld from the search) that is portable to other domains.

Real-world applications:

• Drug repurposing triage. Prioritizing existing compounds for cancer types where their targets show selective dependency, with a mechanistic rationale attached that a research team can review.
• Hypothesis ranking in biomedical labs. Using comparative LLM judgment to filter a large space of mechanistic explanations down to a shortlist worth the cost of wet-lab follow-up.
• Traceable research audit trails. The recorded parentage and operator provenance for each hypothesis allow reviewers to see which claims were combined, revised, or discarded, and on what basis.
• Transferable discovery pipelines. The genetic control flow could be paired with task-specific representations and evidence sources in areas where agents already operate, such as single-cell analysis, cell-type annotation, or materials simulation.

Industry relevance: the cost profile matters. HypoEvolve reaches its results at roughly $0.56 and 206 model calls per run — cheaper than two of the six baselines despite searching a larger candidate pool — which places iterative agentic search within reach of pharmaceutical research groups rather than only well-resourced AI labs. The finding that adding model calls does not consistently improve results is directly useful for teams deciding how to budget inference.

Future Directions

• Reducing the quadratic cost of pairwise comparison. Scoring consumes 167 of 206 model calls per run and grows with the square of the pool size. Sparse or scheduled comparison strategies could redirect that budget toward larger populations or longer search horizons.
• Grounding fitness in something beyond LLM judgment. Because selection depends on model-generated assessments, unsupported but persuasive hypotheses can propagate across generations. Incorporating expert preferences or domain evidence directly into the comparison step is a natural next step.
• Isolating crossover and mutation more rigorously. The operator ablation covers only eight cancer types and produced no result surviving multiple-testing correction, leaving the individual contributions of each variation operator unresolved; adaptive rates and diversity-aware replacement are suggested as extensions.
• Testing transfer beyond drug repurposing. Whether the same control flow works across tasks and model families — and how it pairs with agentic sequential falsification, where free-form hypotheses are routed toward concrete external tests — remains open.

Target Audience

Researchers and practitioners working on LLM-based scientific agents and automated discovery, particularly those interested in how multi-agent coordination is structured rather than only in what individual models can do. It is also aimed at computational biologists and translational researchers evaluating AI-generated drug repurposing hypotheses, and at machine learning engineers designing evolutionary or population-based search over language artifacts. Readers who want a concrete, statistically rigorous example of "collaboration design as an experimental variable" will get the most from it; readers looking for validated therapeutic findings will not, since the authors are explicit that all generated hypotheses require experimental confirmation.

Authors’ abstract

Scientific agents contribute to hypothesis discovery by synthesizing evidence, assessing proposals, and developing new explanations. Recent systems combine scientific agents with evolutionary search through critique, comparison, and revision. However, how different forms of agent collaboration affect hypothesis quality remains an open question. Answering this question requires separating the effects of agents' scientific capabilities from those of their collaboration. A framework must therefore preserve agents' scientific roles and support rules for combining, revising, and retaining hypotheses. Building on this view, we introduce HypoEvolve, which makes collaboration explicit through successive updates to a hypothesis population. Specifically, we propose a generational genetic algorithm to coordinate specialized large language model (LLM) agents that integrate mechanistic arguments, reconsider assumptions, and assess evidence and testability. Each generation specifies how scientific judgments and new proposals reshape the population, making collaboration effects on hypothesis quality directly testable. Moreover, we design our evaluation around scientifically meaningful hypotheses that explain how a proposed intervention could work. Drug repurposing links these explanations to target-level biological claims assessed against external evidence. Specifically, we adapt DepMap and Open Targets into complementary external measures grounded in experimental, genetic, and clinical evidence. Across 34 cancer types, HypoEvolve achieves the highest scores against six baselines on both measures. DepMap selectivity reaches 0.171, versus 0.115 for the strongest baseline. Gains over single-pass generation also generalize to held-out cancer types. HypoEvolve advances a vision of autonomous science in which AI research teams achieve a capacity for discovery beyond that of individual models.