Predicting Unseen Scientific Interactions: Conditional Flow Models Hallucinate Non-Additive Effects

AI summary

Overview

Research area: Generative machine learning for scientific prediction — specifically conditional flow-matching and diffusion models used to predict biological and physiological responses under experimental conditions that were never observed during training.

Technical level: Advanced. The paper assumes familiarity with conditional generative models, flow matching, functional ANOVA (fANOVA) decomposition, and interaction contrasts from experimental design.

Scope: The paper evaluates whether conditional flow models can predict genuine non-additive interactions (synergy, antagonism, epistasis) between two conditions for held-out targets, and it introduces a decomposition that separates true interaction recovery from hallucinated interaction structure.

What This Paper Is About

Scientific questions frequently ask whether one intervention changes the effect of another — whether two drugs interact, or whether two gene perturbations combine non-additively — but these interactions usually explain only a small fraction of total variation in the data. Because aggregate fidelity metrics such as MSE or FID are dominated by the much larger main effects of individual factors, a model can look accurate overall while getting the scientifically important interaction wrong. The authors ask how flow-matching models learn interactions, when they extrapolate correctly to unseen conditions, and when they instead generate plausible-looking but fabricated non-additive structure.

Key Contributions

1. Diagnostics for interaction prediction. The authors decompose a predicted interaction into a component along the true interaction (recovery, denoted alpha) and an orthogonal component (hallucination, denoted beta), with the exact identity that normalized interaction error squared equals (1 − alpha)² + beta². A zero-interaction predictor scores (alpha, beta) = (0, 0) with error 1.

2. Demonstration that models hallucinate non-additivity. In the PPG simulator with controlled interaction strength, flow models generate interaction structure even when the true interaction has been completely removed (gamma = 0), and hallucination grows as targets become less identifiable.

3. Continuation, not distance, determines extrapolation. Held-out regions of the same condition space that sit at equal distance from the training support can be recovered almost fully or barely at all, depending on whether the condition structure continues beyond the training support.

4. Aggregate metrics conceal interaction failures. Endpoint error and interaction error follow separate learning trajectories, low endpoint errors can accompany failed interaction recovery, and recovery in the output space can break down after applying a nonlinear scientific readout.

Main Findings

• Interactions are learned far later than main effects. In the PPG simulator, additive and endpoint errors appear largely converged by 10K optimizer steps while interaction error has not; for seen targets the model needs at least an order of magnitude longer training. For unseen targets, interaction error for the held-out low-low (LL) corner stays high and plateaus by 50K steps, while for the high-high (HH) corner it is approximately 7 times smaller by 20K steps.

• Endpoint-based checkpoint selection can be actively harmful. For held-out Norman double perturbations, interaction error begins to increase after roughly 30K steps while endpoint error continues to decrease until roughly 50K steps. In the well-constrained dose-response setting, no clear decoupling is observed.

• Two geometrically matched holdouts behave completely differently. At full interaction strength (gamma = 1), the joint model recovers HH targets almost perfectly (alpha approximately 1.00) but only weakly recovers LL targets (alpha approximately 0.31), with hallucination much higher for LL (beta approximately 0.43 versus 0.08).

• The hallucinated structure scales with reduced identifiability. As less local support is available, hallucination increases across systems: PPG HH beta 0.12 → 0.25, dose-response 0.18 → 0.24, perturbations 0.80 → 0.86, conformers 0.63 → 0.69 → 0.79.

• Recovery also degrades with harder identification, but at very different rates. PPG LL recovery drops 0.30 → 0.05, perturbations 0.66 → 0.41, while dose-response barely moves (0.98 → 0.97) and a standard deterministic regression baseline achieves nearly identical recovery (0.98 → 0.97). Molecular conformers are an exception, with alpha increasing as local support shrinks (0.69 → 0.70 → 0.76), which the authors attribute to exaggerated between-scaffold-family differences while within-family recovery stays attenuated.

• Models fabricate interaction when none exists. At gamma = 0, all models show beta^(1) > 0, with joint-model hallucination beta^(1)_LL approximately 0.19 and beta^(1)_HH approximately 0.08, suggesting the inductive bias produces a baseline level of non-additivity.

• A scientific readout can destroy interaction recovery. For unseen HH targets the joint model predicts the waveform interaction well (alpha^wf approximately 1.00, beta^wf approximately 0.07), but after the AC/DC pulse-amplitude readout recovery halves (alpha^AC/DC approximately 0.50) and hallucination explodes (beta^AC/DC approximately 1.90).

• Aggregate accuracy can persist while interaction prediction fails completely. For PPG LL, enlarging the held-out region keeps endpoint error non-trivial but low (E_end = 0.104 ± 0.006 at 25%, 0.144 ± 0.0004 at 50%, 0.165 ± 0.007 at 75%) while interaction error becomes no better than predicting zero interaction (E_I = 0.798 ± 0.065, 0.960 ± 0.009, 0.992 ± 0.031 respectively). In the same table, PPG HH goes from E_end = 0.006 ± 0.001 / E_I = 0.125 ± 0.035 (25%) to 0.027 ± 0.002 / 0.251 ± 0.026 (75%), perturbations from 0.582 ± 0.027 / 0.919 ± 0.018 (supported) to 0.561 ± 0.021 / 1.052 ± 0.044 (cold-start), and conformers from 0.436 ± 0.003 / 0.704 ± 0.005 (neighborhood) to 0.513 ± 0.003 / 0.828 ± 0.005 (distant).

• Interactions are small relative to total signal. Interaction-to-total norm ratios are 5.6% (PPG), 23.2% (dose-response, median assay), 46.1% (perturbation, median pair), and 47.4% (conformer, median family), corresponding to 0.3%, 5%, 21%, and 22% of response energy respectively.

• Small interaction error does not imply accurate conditional means, and vice versa. The authors show analytically that interaction error equals e_AB − e_A − e_B + e_0, so the four endpoint errors have no guarantee of cancelling.

Methodology in Plain English

The authors train conditional flow-matching models that learn a velocity field transporting a standard normal distribution to the conditional data distribution, following the approach of Lipman et al. (2023), with minor per-domain variations. For molecular conformers they use a published equivariant flow-matching model conditioned on molecular graphs (Hassan et al., 2024).

For every joint condition AB they compare three prediction strategies: a Joint model trained and evaluated directly on AB, a Composed model that sums the learned conditional velocity fields for A and B and subtracts the baseline field at every sampling step, and a Shared-model AB control that uses the same network as the Composed evaluation but queries AB directly, isolating the effect of the compositional step itself.

Interaction is defined via the standard additive contrast — the AB response minus the A response minus the B response plus the baseline — or, for continuous factors with no natural reference level, via a functional ANOVA decomposition over a condition grid. For drug combinations, additivity is replaced by Bliss independence because fractional effects are bounded to [0, 1], so the interaction is the deviation of the observed combined effect from the Bliss-predicted effect; positive values indicate synergy and negative values antagonism. The authors note that Bliss independence is equivalent to additivity after the log(1 − E) readout, illustrating that interactions depend on representation.

Model predictions are evaluated both as "anchored" estimates (using true reference conditions, isolating error on the unseen condition) and "full" estimates (using the model for every term). They also distinguish three identification regimes for held-out targets: smoothness-identified (the held-out region is surrounded by joint observations), relationally-identified (each factor is observed in other combinations), and underdetermined (only marginals are observed).

Four systems are studied. The PPG simulator is a deterministic replay of a fixed Monte Carlo photon-path bank using a three-layer skin model simplified from PPGen (Behrmann et al., 2026), with 10^8 photon histories for 35 scattering settings, mapping epidermal melanin (0.25–14%) and dermal blood volume (0.1–3.9%) to 16-channel PPG waveforms with 64 timesteps across four source-detector distances (3–6 mm) and four wavelengths (525, 660, 850, 940 nm). A 101 × 101 Mel-BV2 grid is decomposed by balanced fANOVA into grand mean plus main effects plus interaction, and a controlled interaction-strength grid is built as Y_gamma = Y_tilde + gamma·I; 10^6 Mel-BV2 pairs are drawn and waveforms obtained by bilinear interpolation. Holdout regions are nested LL or HH rectangles at 25%, 50%, and 75% (735k/15k, 490k/10k, and 245k/5k training/validation samples). The physiological readout is the standard normalized pulse-amplitude AC/DC ratio per channel.

The other three datasets are the Cokol et al. (2011) yeast drug-pair screens (200 drug-pair assays across 33 drugs, 175 different-drug and 25 same-drug controls, each an 8 × 8 dose grid of 96-dimensional growth curves from 15-minute measurements over 24 hours, 12800 trajectories total; 2 × 2 and 3 × 3 blocks held out), the Norman et al. (2019) K562 CRISPRa Perturb-seq dataset (91205 cells, 5045 genes, predicted in 500-dimensional PCA space with graph-supported and cold-start holdouts), and GEOM conformer ensembles (Axelrod and Gomez-Bombarelli, 2022) using chemical squares with two edits on a shared core and neighborhood, parents-only, or distant training supports.

Why This Matters

The paper argues that standard aggregate fidelity metrics can hide whether a generative model has learned the scientific effect of interest, which risks experiments that look computationally validated but answer the wrong question. It recommends that scientific evaluation test interaction recovery directly in the representation used to draw conclusions, and that individual factors alone are insufficient — either joint observations or appropriate structural assumptions are needed to constrain unseen interactions.

Real-world applications:

• Drug combination screening: predicting synergy or antagonism for dose combinations that were never assayed, where Bliss-independence deviations are the clinically relevant quantity.
• Wearable and optical sensing: the PPG simulator studies how pulsatile optical change depends jointly on melanin and dermal blood volume, which is directly relevant to skin-tone-dependent bias in pulse oximetry and wearable photoplethysmography.
• Single-cell perturbation studies: predicting responses to gene pairs, where the cell-type-by-perturbation interaction is often the central question but explains less variance than either factor alone (as cited from Schäfer et al., 2026).
• Molecular design: predicting conformer properties such as radius of gyration for double-edited molecules where no direct training example exists.

Industry relevance: pharmaceutical companies running combination therapy programs, biotech and computational drug discovery groups relying on perturbation prediction models, wearable and medical device developers, and ML teams whose model selection currently depends on aggregate error alone.

Future Directions

• Extending the diagnostic decomposition beyond the two-factor interactions studied here, which the authors list as a limitation.
• Moving from mean-based contrasts to distributional interaction measures, the second stated limitation of the work.
• Determining when smoothness-based, relational, or explicit structural assumptions can substitute for direct joint observations, given that identifiability regime — not geometric distance — governed extrapolation success in these experiments.
• Developing training or model-selection procedures that monitor interaction recovery alongside aggregate error, since endpoint-based checkpoint selection picked models whose interaction predictions had already degraded on the Norman perturbation data.

Target Audience

Researchers and practitioners working on conditional generative models for scientific data — perturbation prediction, drug combination modelling, physiological simulation, and molecular generation — along with method developers interested in evaluation beyond aggregate fidelity metrics and biostatisticians familiar with interaction contrasts and functional ANOVA. The paper assumes comfort with generative modelling terminology, so it is best suited to readers at an intermediate-to-advanced technical level.

Authors’ abstract

Conditional generative models are increasingly used for scientific prediction beyond the direct training support. Yet scientific questions often concern whether one intervention changes the effect of another. Such interactions can be of primary interest while explaining little of the total variation, allowing models to appear accurate overall without recovering the relevant scientific effect. It remains unclear how these models learn interactions and when these successfully extrapolate to unseen conditions. We therefore study interaction recovery in flow-matching models using a physiological time-series simulator, drug dose-response data, combinatorial cell perturbations, and molecular conformer generations, where we separate recovery from hallucinated interaction structure. In the controlled setting, individual effects are learned much earlier than interactions and models hallucinate interactions where none exist. Across settings, we find qualitatively different generalization problems, with successful prediction depending on whether condition structure continues from observed to held-out targets. When it does, interactions are recoverable. If not, models can generate plausible responses while missing the interaction. We furthermore find that hallucinations increase as targets become less identifiable, and that successful interaction recovery in output space can fail under nonlinear readouts. Finally, we show that aggregate fidelity metrics can conceal whether models learn and extrapolate interactions.