Principia: Test-Time Hypotheses to Principle Construction for Scientific Discovery

AI summary

Overview

Research area: Machine learning for scientific discovery, specifically LLM-driven construction of scientific knowledge (causal hypotheses and general principles) from experimental observations gathered during a discovery run.

Technical level: Intermediate. Readers should be comfortable with the idea of an oracle-guided optimization loop, retrieval-augmented generation, and standard classification metrics, but the paper's core ideas are conceptual rather than mathematically heavy.

Scope: The paper introduces and evaluates Principia, an LLM framework that builds comparative causal hypotheses from pairs of related inputs, generalizes them into principles by clustering, and revises them as new experimental observations arrive, tested across molecular optimization, protein engineering, gene perturbation, and molecular property prediction.

What This Paper Is About

In real scientific discovery, researchers rarely know the underlying principle in advance; they collect observations, propose explanations for individual outcomes, and generalize recurring patterns into broader rules. This translation from accumulated experiments to principles is slow and requires heavy domain expertise, and existing LLM-based discovery systems largely skip it, using only internal knowledge, retrieved literature, or flat textual summaries of past experience. Principia targets that gap by making the hypothesis-to-principle step an explicit, iterative component of test-time discovery.

Key Contributions

1. A hypotheses-to-principle framework for test-time discovery. Principia builds structured, reusable scientific knowledge from observations accumulated during the discovery process itself, rather than relying on pretrained knowledge or external sources.

2. Comparative causal hypothesis generation. Instead of explaining a single input-outcome pair, the framework pairs related inputs (via modification history in optimization, or fingerprint similarity in prediction) and asks an LLM to explain how their similarities and differences account for their differing or matching outcomes.

3. Inductive principle construction via similarity-based clustering. Hypotheses are grouped with k-medoids clustering over domain-specific distances, and an LLM induces a compact principle from the recurring relationships in each cluster, stored in a separate principle memory.

4. Hypothesis revision from new observations. Retrieved hypotheses guide the next predictions or candidate proposals; the resulting observations are then used to revise those hypotheses and reconstruct the principles, closing an observation-hypothesis-application-revision loop.

Main Findings

• Molecular optimization: Principia reaches an averaged Avg@100 of 0.340 and AUC@10 of 0.279 across JNK3, GSK3β, Median1, Median2, and Valsartan Smarts, versus 0.254/0.250 for the best experience-summary baseline (ExLLM) and 0.235/0.222 for the unguided optimizer (MolLEO).

• Protein and gene tasks: Principia attains the best average AUC@10 (0.605) on Syn-3bfo, AAV, GB1, Scharenberg22, and IFNG, including an Avg@50 of 0.398 on GB1 versus 0.236 for ExLLM, and a Syn-3bfo Avg@50 of 0.084 versus −0.286 for ExLLM.

• Structured knowledge beats flat summaries and external knowledge: Principia outperforms baselines using experience summaries (ExLLM), web retrieval (WebSearch), search-plus-experience (DeepEvolve), and a variant without observation-guided causal hypotheses (PiFlow), which often underperforms even the unguided MolLEO.

• Molecular property prediction: Across eight TDC binary classification tasks with a scaffold split, Principia averages 69.75 accuracy and 69.03 macro-F1, against 63.95/62.30 for the strongest few-shot baseline (k=7) and 63.26/61.03 for a textual-summary baseline. Gains hold on molecules with scaffolds absent from the training set.

• Hypotheses carry real signal: Adding hypothesis guidance raises GB1 AUC@10 from 0.245 to 0.337 (+0.092) and Syn-3bfo from −0.019 to 0.116 (+0.135); comparative hypotheses also outperform a non-comparative variant that explains single observations.

• Both components matter: Ablating principle construction lowers performance, and ablating hypothesis revision lowers it too; removing LLM-based knowledge retrieval drops AUC@10 to 0.364, close to the 0.374 obtained without revision, indicating selective retrieval is what makes revision pay off.

• Budget efficiency: On Syn-3bfo, Principia matches the final performance of ExLLM, WebSearch, MolLEO, and DeepEvolve with roughly 55%, 84%, 86%, and 86% fewer oracle evaluations, respectively.

• Scientific plausibility: In a qualitative example, the induced principle identifies epoxide/oxirane-bearing aryl ether motifs as a likely mutagenicity driver and distinguishes tolerating aminoalcohol side chains, consistent with prior literature linking epoxide-containing compounds to mutagenicity.

Methodology in Plain English

The framework treats discovery as a loop with two memories: a graph-structured hypothesis memory whose nodes are inputs and whose directed edges store a comparison plus its explanation, and a principle memory holding cluster-level generalizations.

1. Build comparative evidence. For optimization, each generated candidate is compared against the input it was derived from, together with both oracle scores. For prediction, each new molecule is paired with up to three previously observed molecules chosen by Tanimoto similarity, without filtering on label, so comparisons can be same-label or different-label.

2. Generate a causal hypothesis. An LLM receives the two inputs, their outcomes, and the task description, and writes a short natural-language explanation. If outcomes differ, it points at structural differences that could explain the gap; if they match, it points at shared features that could preserve the outcome.

3. Induce principles. Inputs in the hypothesis graph are clustered with soft-balanced k-medoids using domain-specific distances (Tanimoto on 2,048-bit Morgan fingerprints with radius 2 for molecules, normalized Hamming or 3-mer Jaccard for proteins, cosine distance on text-embedding-3-large embeddings for gene symbols). An LLM then reads all hypotheses attached to a cluster and writes one compact principle capturing recurring relationships, including positive and negative patterns.

4. Apply and revise. For a new query, an LLM selects relevant hypotheses and principles from candidate pools and uses them to predict a property or propose a modified candidate. The resulting observations are fed back: each used hypothesis is revised in light of the new evidence, new hypotheses are generated for new inputs, and principles are reconstructed from the updated set.

The backbone models are GPT-5-mini for the optimization experiments and GPT-5.4-mini for molecular property prediction. Optimization results are averaged over three runs; prediction results over three random seeds. The paper does not report the exact oracle budgets or the numeric values of the retrieval counts k_h and k_p in the main text.

Why This Matters

Impact on research. Most LLM-for-science systems either retrieve static external knowledge or compress experience into a prose summary. Principia shows that the intermediate representational layer scientists actually use, explicit causal hypotheses generalized into principles, can be operationalized and gives measurable gains on held-out molecules and across protein and gene tasks. It also reframes experimental observations as knowledge to be structured and reused, not just as feedback signal for the next step.

Real-world applications:

• Drug discovery: designing molecules that hit activity targets (JNK3, GSK3β) while screening out liabilities such as DILI, hERG, Ames, and skin reaction toxicity.
• ADMET and safety triage: predicting CYP inhibition and toxicity endpoints for compounds with scaffolds not seen in training.
• Protein engineering: optimizing protein sequences for target functions, as in the AAV, GB1, and Syn-3bfo tasks.
• Gene perturbation and cellular screening: identifying perturbations that produce a desired cellular response, with fewer wet-lab evaluations needed.

Industry relevance. The reported reduction in oracle evaluations (about 55% versus an experience-summary baseline on Syn-3bfo) speaks directly to cost, since each evaluation corresponds to a wet-lab or simulation experiment. Pharma, biotech, and materials teams running expensive design-make-test cycles are the natural beneficiaries, and the approach is model-agnostic enough to sit on top of existing LLM-based design pipelines.

Future Directions

• Scaling and validating principle quality. The paper offers one worked qualitative example consistent with mutagenicity literature; broader, systematic checks of whether induced principles hold up against expert review or curated scientific knowledge remain open.

• Richer memory and revision policies. The comparison structure is currently pairwise and edge-local, and revision is triggered for hypotheses retrieved in the current round; how to prioritize which hypotheses to revisit, or handle conflicting principles, is not settled.

• Transfer and reuse across tasks. Principles are constructed per test-time task. Whether induced principles can be carried across related objectives or domains, and when they should be discarded, is an open question.

• Dependence on backbone and retrieval choices. Results are reported with specific GPT-5-family backbones and LLM-based retrieval; sensitivity to smaller or open models, and the exact retrieval counts and budgets, are not reported and would matter for practical deployment.

Target Audience

Researchers and practitioners working on LLM agents for scientific discovery, automated molecular and protein design, and retrieval- or memory-augmented LLM systems. It is also useful for computational chemists and biologists who want to understand how structured, interpretable knowledge can be extracted from iterative experimental loops, and for ML engineers building design-make-test pipelines where evaluation cost is the binding constraint.

Authors’ abstract

Scientific discovery requires identifying the scientific principles underlying observed phenomena to guide further investigation. As these empirical principles are often unknown in complex discovery processes, they are constructed via a hypotheses-to-principle pipeline to obtain recurring causalities from progressively accumulated observations during test-time discovery. However, constructing this knowledge requires substantial domain expertise, time, and effort, making this translation stage a bottleneck for scientific discovery. To address this limitation, we propose Principia, an LLM-based framework that constructs and continually refines scientific principles at test time. Drawing on how human researchers develop principles, Principia forms comparative causal hypotheses from related inputs and their observed outcomes, then synthesizes recurring relationships across these hypotheses into inductive principles. By applying the constructed hypotheses and inductive principles to the task, it obtains new observations, revises the hypotheses used based on these new observations, and reconstructs principles from the updated hypotheses, allowing scientific knowledge to continuously evolve throughout test-time discovery. Across benchmarks spanning molecular optimization, protein engineering, gene perturbation, and molecular property prediction, Principia outperforms baselines relying on internal knowledge, external sources, or experience summaries, while generating principles that are plausible and consistent with the scientific literature, demonstrating its effectiveness in constructing and refining task-specific hypotheses and principles at test time for scientific discovery. Our code is available at https://github.com/DongkiKim95/Principia.