LLM-to-Flow Distillation: Teaching Natural Language Goals to Scientific Generators

AI summary

Overview

Research area: Machine learning for scientific discovery — specifically, aligning domain-specific generative models (flow and diffusion models over peptides and molecules) to high-level natural-language goals using a frozen large language model (LLM) as a teacher.

Technical level: Intermediate. The paper assumes familiarity with generative modeling, preference optimization (DPO), and LLM prompting, but its central idea is conceptually simple and clearly explained.

Scope: The paper introduces LLM-to-Flow Distillation (LFD), an algorithm that lets a frozen, API-only LLM construct an adaptive curriculum of intermediate sub-goals and distill its ordinal judgments into a scientific generator, and evaluates it on therapeutic peptide and small-molecule design tasks without any proxy reward.

What This Paper Is About

Scientific discovery needs two things that are usually split across two model families: high-level reasoning about chemistry and biology (held by LLMs, which cannot reliably generate valid designs) and the ability to sample structurally valid designs (held by flow and diffusion models, which cannot interpret a scientist's natural-language goal). The paper asks how to align a domain-specific scientific generator directly to a natural-language goal such as "generate peptides with promising anticancer activity" without defining an explicit reward function, which is often unavailable or unreliable in experimental science.

Key Contributions

1. A formal problem statement for language-goal alignment. The authors define a natural-language goal g as inducing a discovery region Ω_g = {x : f_g(x) = 1} in a design space, and cast alignment as fine-tuning a pre-trained flow model v_θ0 into v_θ such that p_θ(Ω_g) ≫ p_θ0(Ω_g), given only a frozen LLM, observable descriptors of generated designs, and no scalar proxy reward.

2. The LFD algorithm. An LLM acts as an adaptive teacher that inspects the generator's current samples, proposes an attainable intermediate sub-goal that relaxes the final goal (g_t ← SetGoal(g, g_{t-1}, O(X_t))), labels candidates as positive, negative, or abstained, and distills the resulting preferences into the generator with DPO. Each round regenerates preferences from the current student, using p_θt as the reference, so updates track the student's evolving "generable set."

3. An empirical study of LLM reliability in scientific judgment. Across three scientific domains and three frontier LLMs, the authors show that absolute LLM scores are miscalibrated but their relative judgments carry signal, motivating ordinal (rather than cardinal) supervision.

4. Evaluation on peptides and molecules. LFD is compared against classifier Guidance, DPO, Flow-GRPO, and DiffusionNFT on two peptide goals and three molecular goals, and outperforms them on held-out activity and docking evaluations despite never accessing the proxy rewards those baselines optimize.

Main Findings

• Absolute LLM scores are unreliable, relative judgments are not. Three API-accessible LLMs — Gemini 3.7 Flash, GPT 5.6 Terra, and Claude Opus 5 — were tested on (i) peptides from PepDFM judged by alpha-helical hydrophobic moment, (ii) crystal structures from MatterGen judged by energy above the convex hull after MatterSim relaxation, and (iii) QM9 small molecules judged by MolLogP. On a 1–100 scale, estimated scores were far from ground truth, but all four ordinal elicitation mechanisms (independent 1–100 scoring, listwise ranking, batch classification as positive/negative/abstain, and direct pairwise comparison) recovered pairwise accuracy above the 50% chance line. Duels scale quadratically in samples while the other three scale linearly, so LFD uses batch classification.

• Anticancer peptide goal (Goal 1). Using PepDFM as the discrete flow generator, LFD reached the highest held-out AntiCP2-alternate score in Table 1 — 53.54% ± 3.32 (reported in the text as 53%) — versus 35.94% ± 2.33 for Guidance and 26.43% ± 2.91 for the pre-trained model. Its HemoPI2 toxicity score was 42.48% ± 2.04, and its distance from the pre-trained amino-acid prior was the largest at 0.31 ± 0.03. The curriculum spanned seven sub-goals with one to four rounds each plus two final rounds on the goal.

• Reward hacking is mitigated. Guidance saturated its ACP proxy at 100% but reached only 35.94% on the held-out AntiCP2-alt evaluation. LFD never optimized either anticancer predictor and still achieved the best held-out score.

• Cell-penetrating peptide goal (Goal 2). LFD raised the pLM4CPPs score from 8.81 ± 3.20 to 69.06 ± 6.36, versus 57.04 ± 6.22 for the best baseline, DiffusionNFT, while HemoPI2 fell from 40.84 ± 1.96 to 37.02 ± 1.69 — the lowest predicted hemolysis among compared methods. Training used g_0, g_1, and the original goal for three, two, and two rounds respectively, and never accessed the ESM2-based pLM4CPPs evaluator.

• D2 dopamine receptor goal (Goal 3). With FlowMol3 pre-trained on GEOM-Drugs, LFD produced the best mean on all three held-out evaluators: GNINA CNNaffinity rose from 6.11 ± 0.33 to 6.54 ± 0.09 pK (above Flow-GRPO's 6.32 ± 0.06), Vina improved from −7.52 ± 0.39 to −8.48 ± 0.16 kcal/mol (versus −7.90 ± 0.23 for DiffusionNFT), and rDock improved from −7.64 ± 1.33 to −9.89 ± 0.38 (versus −9.39 ± 0.26 for the best baseline). Validity was 98% and PoseBusters conformer quality 89.5%.

• GSK3β goal (Goal 4). LFD again led on all three evaluators: GNINA from 5.80 ± 0.17 to 6.77 ± 0.11 pK, Vina from −7.32 ± 0.19 to −8.84 ± 0.16 kcal/mol, and rDock from −8.51 ± 0.55 to −12.87 ± 0.57, against baselines optimizing a proxy TDC GSK3β classifier. RDKit validity was 96% and PoseBuster conformer quality 84%. LFD surpassed all baseline evaluations after 3 rounds.

• Complex structural constraints via natural language (Goal 5). For a goal requiring a six-membered aza-aromatic ring with a free primary amino group adjacent to a pyridine-like nitrogen plus a distinct fully saturated ring with a free secondary amine, the pre-trained model produced no hits over 38,500 samples. LFD-adapted FlowMol3 produced a satisfying hit in a 512-sample batch at round 6 under the third sub-goal g_2. All reward-guided baselines failed even when given an LLM-generated binary reward implementing the full goal.

• Interpretability. The teacher emits inspectable traces. In one example it rejected the peptide MLKGRWPDEQKKNHLMNYNFMLIVFVLTFNISFQSSNI (charge index +2.1, hydrophobicity 47.4%, lower-third non-hemolysis band) because the contiguous hydrophobic tract MLIVFVL violated the operative sub-goal, and flagged in vitro hemolytic threshold and aggregation state as sources of uncertainty.

Methodology in Plain English

The authors keep two models fixed in place. A pre-trained scientific generator (a flow model that turns noise into peptide sequences or 3D molecules) is the student. A frozen LLM accessed only through an API is the teacher — no weights, gradients, or fine-tuning.

Rather than asking the LLM for a numeric reward, the loop works in rounds. In each round the student samples a batch of designs, and those designs are converted into observables the LLM can read: for peptides, the amino-acid sequence plus charge, hydrophobicity, aromaticity, amphipathicity, and coarse developability bands; for molecules, the SMILES string plus descriptors such as molecular weight, hydrogen-bond counts, topological polar surface area, and rotatable-bond counts. The teacher looks at those observables, infers what the student can currently produce, and writes a sub-goal — a broad relaxation of the final goal that is scientifically connected to it and easy enough that the current samples contain both good and bad examples. The sub-goal is accepted only if the teacher finds enough positive and negative candidates to tell apart.

Under that sub-goal the teacher labels each candidate positive, negative, or abstain, and every positive-negative pair becomes a preference. These preferences are fed to standard DPO, with the current student acting as its own reference model so each update stays close to where the model is now. Because the batch is resampled every round and the reference is refreshed, the teacher's judgments always track the student's evolving capabilities. The analogy the authors draw is to annealing: a hard optimization problem is approached through a sequence of easier ones that tighten toward the target — but unlike classical annealing, the schedule is not fixed in advance; the LLM writes it online. Each round is a KL-regularized step toward an implicit target p*_t ∝ p_θt · exp(f_gt / β), and terminal log-densities are optimized through a denoising-loss surrogate.

Why This Matters

Impact on research. The paper decouples scientific goal specification from reward engineering. It shows that when no trustworthy proxy reward exists — a common situation in chemistry and biology, where a goal like anticancer activity requires long, expensive experiments — an LLM's first-principles reasoning can still supply a dense enough learning signal, provided it is used ordinally and staged through a curriculum. It also offers a concrete diagnostic: reward-guided optimization can saturate a proxy at 100% while held-out performance stalls near chance, and language-guided adaptation avoids that failure mode.

Real-world applications.

• Therapeutic design. Generating peptides with anticancer or cell-penetrating activity while limiting hemolysis, using only sequence and physicochemical descriptors.
• Structure-based drug discovery. Generating molecules with strong docking profiles against specific targets such as the D2 dopamine receptor or the GSK3β ATP pocket without a docking reward in the loop.
• Specification-heavy discovery campaigns. Encoding structural constraints — ring systems, saturation, functional-group placement, adjacency rules — that would otherwise require bespoke reward code, expressed instead in plain language.
• Human-in-the-loop scientific tooling. Inspectable sub-goals and per-candidate rationales let a scientist audit and potentially intervene in the adaptation process.

Industry relevance. Pharmaceutical and biotech research groups increasingly run flow- and diffusion-based generative models but lack reliable in-silico reward functions for the properties they care about most. A method that converts a scientist's written goal into an adaptation signal using only an off-the-shelf API model could shorten the loop between hypothesis and candidate generation, reduce reliance on hand-built reward models, and lower the risk of optimizing against a proxy that does not generalize.

Future Directions

• Beyond the two demonstrated domains. The paper studies peptides and small molecules. Whether the curriculum mechanism transfers to materials, proteins, or nucleic acids — the paper mentions material design as an analogous setting — is left open.
• Teacher reliability bounds. Only three LLMs (Gemini 3.7 Flash, GPT 5.6 Terra, Claude Opus 5) were probed for scientific judgment quality. How LFD degrades with weaker teachers, or how it behaves when the teacher's relative ordering is wrong rather than merely miscalibrated, is not reported.
• Sensitivity to curriculum design. The rounds allotments differ by task (for example, seven sub-goals for the anticancer goal; five rounds on g_0 for D2). The paper does not report a systematic study of how these allocations or the threshold for accepting a sub-goal affect outcomes.
• Cost and compute accounting. Although the paper notes that batch classification avoids the quadratic API cost of pairwise duels, no wall-clock time, token budget, or training compute comparison against the reward-guided baselines is reported in the content available, and no limitations section is included.

Target Audience

Researchers and practitioners working at the intersection of generative modeling and scientific discovery — particularly those training flow or diffusion models for molecular or biological design who lack a reliable reward function. It will also interest groups studying LLM-as-judge methods, preference optimization applied to non-text domains, and curriculum or annealing-based fine-tuning. Readers without background in probabilistic generative models may find the formal sections (the discovery-region formulation and the KL-regularized interpretation of each round) the most demanding, but the experimental case studies are self-contained and readable on their own.

Authors’ abstract

Generative discovery in science requires coupling high-level scientific reasoning with the ability to generate samples respecting the structure of a design space. Yet these capabilities remain largely separated: foundation LLMs encode broad chemical and biological knowledge, while domain-specific flow- and diffusion-based generators capture the priors needed to produce valid sequences and geometries. We study how to align such scientific generators to high-level natural-language discovery goals without requiring an explicit proxy reward, which is often unavailable or poorly captured by learned predictors in experimental sciences. We introduce LLM-to-Flow Distillation (LFD), where a frozen LLM acts as an adaptive teacher for a domain-specific generator. Given a goal such as 'generate peptides with promising anticancer activity', the teacher constructs a curriculum of intermediate sub-goals aware of the flow's current capabilities, and progressively directs them toward the original discovery goal. At each round, the LLM judges sampled candidates under the current objective. These judgments induce preferences that are iteratively distilled into the generator through DPO. This procedure provides dense learning signals even when the initial model places negligible probability on designs satisfying the final goal, enabling adaptation toward poorly covered, out-of-distribution regions. Experiments on therapeutic peptides and small-molecule structure design demonstrate that LFD enables a new form of generative control: aligning arbitrary domain-specific scientific generators to scientific discovery tasks expressed via high-level natural-language goals.