Opening the Black Box of Classifier-Free Guidance via Information Bottleneck

AI summary

Authors: Jiayang Gao, Tianyi Zheng, Jiayang Zou, Fengxiang Yang, Luyao Fan, Lv Tang, Bo Li, Jia Wang arXiv: 2610.13369v1 [cs.LG], 09 Oct 2026 Affiliations: Shanghai Jiao Tong University; vivo BlueImage Lab, vivo Mobile Communication Co., Ltd.; Stanford University; Michael Smith Laboratories, University of British Columbia

Overview

• Research area: Conditional diffusion and flow-based generative models, specifically the theory and scheduling of Classifier-Free Guidance (CFG), analyzed through information theory.
• Technical level: Advanced. The paper derives Fisher-information balance conditions for the diffusion trajectory, states two theorems, and assumes familiarity with diffusion SDEs, score functions, mutual information, and the information bottleneck principle.
• Scope: The paper derives a time-dependent information bottleneck for conditional diffusion, converts it into an observable, training-free criterion for deciding when to switch CFG on during sampling, and validates that criterion on image generation benchmarks across SiT, DiT, Stable Diffusion 3, and FLUX.1.

What This Paper Is About

Classifier-Free Guidance is standard practice for conditional generation, but most methods apply it with a fixed schedule from start to finish, and newer dynamic variants rely on hand-designed schedules. The paper asks a basic question the authors say remains unanswered: how does the informational relevance between the conditioning input and the score function change over the course of sampling, and when is conditional guidance actually most useful? Their answer is that conditional information only becomes strongly useful after a critical stage defined by a Fisher-information balance condition, and they turn that condition into an online activation rule for CFG.

Key Contributions

1. Information-theoretic perspective. The authors formulate the diffusion trajectory as a time-dependent information bottleneck, objective inf_t L(t) = I(X_0;X_t) − β I(X_t;C) with β > 0, and connect the evolution of mutual information to Fisher information. This yields a balance condition characterizing an information-theoretically motivated transition region for CFG activation (Theorem 1 for SDE-based diffusion, Theorem 2 for stochastic flow/interpolation models).
2. Principled activation criterion. From the difference between conditional and unconditional score norms, they derive an observable guidance-gain statistic ΔJ(t) = E[||s_θ(x_t,t,c)||²] − E[||s_θ(x_t,t,∅)||²], together with a rescaled threshold criterion using the surrogate scale β* = β/(1 − η(t)) that improves numerical conditioning and provides a model-dependent way to locate the effective guidance region.
3. Adaptive guidance strategy. They propose a training-free, plug-and-play dynamic CFG rule that determines the activation time online via a threshold crossing, works for both batch-level and single-sample inference, and recovers fixed-interval guidance in the large-batch concentration regime.
4. Extensive empirical validation. Experiments span SiT, DiT, SD3, and FLUX, supported by multi-seed stability analyses, an alternative activation criterion comparison, and high-resolution single-sample inference.

Main Findings

• Guidance is not uniformly useful over time. The information bottleneck analysis shows that the effectiveness of conditional information becomes significantly stronger only after a critical stage characterized by a Fisher-information balance condition. The authors argue that uniformly applying a fixed CFG strategy is therefore suboptimal.
• Closed-form Fisher terms make the condition observable. For the Ornstein–Uhlenbeck diffusion process, J(X_t|X_0) = n/m(t); for the flow/interpolation form X_t = α_t X_0 + σ_t X_1 on t ∈ [0,1], J(X_t|X_0) = n/σ_t². The optimality conditions reduce to explicit equations relating β and expected squared score norms.
• Reformulation stabilizes the threshold. The original parameterization exhibits a much flatter log β–t relationship than the rescaled version on SiT-XL/2, meaning the inverse mapping from β to t* is poorly conditioned. The simplified formulation with β* gives a steeper curve and substantially reduced sensitivity to hyperparameter perturbations.
• Class-conditional gains. On ImageNet-256 (50k samples, 250 inference timesteps), DiT-XL/2 + Ours (ω = 1.6, log β* = 8.0) records FID 2.06, IS 280.9, sFID 4.5 versus FID 2.29 / IS 276.8 / sFID 4.6 for DiT-XL/2 at ω = 1.5 and FID 2.71 / IS 303.2 / sFID 4.5 at ω = 1.6. On ImageNet-512 (10k samples, 100 timesteps), DiT-XL/2 + Ours (ω = 1.7, log β* = 8.5, B = 16) reaches FID 6.49 versus 6.81 for the ω = 1.5 SDE baseline, with IS 241.4 versus 229.5.
• SiT results. SiT-XL/2 (REPA) + Ours (ω = 1.9, log β* = 5.9, B = 48) achieves FID 1.41, IS 306.6, sFID 4.7, compared with FID 4.67 / IS 388.3 / sFID 5.5 at ω = 1.9 and FID 1.80 / IS 284.0 / sFID 4.5 at ω = 1.35. With the ODE sampler, Ours (ω = 1.8, log β* = 6.5) reaches FID 1.47 and sFID 4.4 versus FID 1.56 and sFID 4.6 for the ω = 1.8, t_h = 0.7 baseline. Interval Guidance (ω = 1.8, t_l = 0, t_h = 0.7) records FID 1.42, IS 305.7, sFID 4.7.
• Text-to-image gains. On SD3 (1024, 28 steps), Ours (ω = 7.5, log β* = 10.0) reaches FID 24.88, CLIP 31.89, and Aesthetic 5.45 versus FID 25.98, CLIP 31.83, and Aesthetic 5.44 for CFG (ω = 7.5, B = 4); average HPSv2.1 rises from 30.33 to 30.55. On FLUX.1 at 256, Ours (ω = 1.5, log β* = 8.0) lowers FID from 30.93 to 30.36 and raises average HPSv2.1 from 29.24 to 29.36, with a slight reduction in CLIP Score (31.10 to 30.93).
• Single-sample inference works. On FLUX.1 at 1024×1024 with ω = 7.5, Ours reaches an average HPSv2.1 of 30.73 with B = 16 versus 20.13 for standard CFG, and 30.29 with B = 1.
• Batch size robustness. On ImageNet-256 (10k samples, 50 inference timesteps) with SiT-XL/2, the SDE baseline gives FID 5.09 / IS 287.4; the method gives FID 4.92 / IS 299.7 at B = 32, 4.89 / 296.1 at B = 16, 4.96 / 294.6 at B = 8, and 4.95 / 294.9 at B = 4.
• Stability across seeds. Over three seeds, the method records FID 4.98 ± 0.06 and IS 298.1 ± 2.6, versus FID 5.18 ± 0.08 and IS 284.1 ± 3.3 for Interval Guidance.
• Activation time concentrates. The mean batch-wise activation time stays at approximately 0.68 across B = 8, B = 16, and B = 32, while the standard deviation falls from 0.107 to 0.086 to 0.054. This matches the paper's argument that the adaptive rule approaches a fixed interval boundary as batch size grows.
• Criterion choice matters. Against a score-discrepancy MSE statistic (FID 5.14, IS 287.5), the Fisher norm difference used by the authors achieves FID 4.92 and IS 299.7.
• Scope limitation. Theorem 2 applies only to stochastic interpolation paths with an independent Gaussian endpoint; it does not directly cover deterministic couplings such as n-rectified flow for n ≥ 2, where the continuous information bottleneck may degenerate. Equation 20 may still be used as a practical heuristic there, but without the same rigorous justification.

Methodology in Plain English

The authors model the generation process as a Markov chain C → X_0 → X_t, where C is the conditioning signal (class label or text prompt) and X_t is the noisy latent at time t. They write a time-dependent information bottleneck objective that trades off how much information the current state retains about the original data against how much it retains about the condition, and then find the time at which this objective is minimized. By expressing mutual information derivatives in terms of Fisher information — which is directly tied to score functions — they obtain a balance condition between conditional and unconditional score magnitudes.

Directly solving the resulting equation is unstable because the expectations involve squared score norms estimated from finite mini-batches, and these are noisy, especially early in sampling. The authors instead renormalize the hyperparameter, defining η(t) as the ratio of marginal Fisher information to noise-kernel Fisher information and setting β* = β/(1 − η(t)). This simplified condition has a much steeper and better-conditioned relationship between the hyperparameter and the solution time.

For the sampling procedure, each reverse step performs a single forward pass with conditional and unconditional inputs concatenated in the same mini-batch, producing batch estimators of the two score norms. Their difference, ΔĴ(t), acts as a proxy for the instantaneous guidance gain. The algorithm scans backward from t = T to 1 and records the latest time step where the gain exceeds the threshold n/(β* m(t)); once that stopping time is found, CFG is turned on for all subsequent steps in the generative process and off before it. The procedure requires no retraining and no external classifier.

Why This Matters

The paper reframes a widely used but heuristically tuned technique as the solution to a well-defined information-theoretic problem. Rather than tuning a schedule by trial and error, practitioners get a criterion tied to measurable score statistics of the model they are already running. The analysis also supplies a common language for existing dynamic guidance methods: Interval Guidance emerges as the large-batch limit of the proposed rule, which explains why manually chosen interval boundaries can work and when they will fail.

Real-world applications:

• Text-to-image generation and creative tooling. The FLUX.1 and SD3 results target prompt-aligned image synthesis, where the work reports better HPSv2.1 and Aesthetic scores and closer adherence to semantic details in prompts.
• Class-conditional image synthesis. The ImageNet-256 and ImageNet-512 experiments apply directly to label-conditioned generation pipelines built on DiT and SiT backbones.
• Inference-time control for deployed systems. Single-sample inference (B = 1) on FLUX.1 at 1024×1024 matters for products where generating a batch of candidates is impractical.
• Model quality tuning without retraining. Because the method is training-free and plug-and-play, it can be layered onto existing pre-trained checkpoints using official weights.

Industry relevance: co-authorship from vivo BlueImage Lab and the use of production-grade models (FLUX.1, SD3, DiT, SiT) indicate direct interest from the image-generation industry, where guidance strength and timing are among the few remaining knobs that can be tuned at inference without retraining costs.

Future Directions

• Deterministic couplings. The theory explicitly does not cover n-rectified flow for n ≥ 2, where the continuous information bottleneck may degenerate. Extending the analysis to deterministic

Authors’ abstract

Classifier-free guidance (CFG) is a standard technique for improving conditional generation in diffusion models, yet its activation schedule is usually selected heuristically. In practice, most existing CFG-based methods rely on a pre-defined activation policy throughout inference, while recent dynamic variants often depend on hand-designed schedules without a principled theoretical explanation of when conditional information is most useful. In order to understand how the informational relevance between conditioning inputs and the score function evolves over the course of generation, we explore when guidance should be activated from an information-theoretic perspective. In this work, we formulate the diffusion trajectory as a time-dependent information bottleneck and show that the effectiveness of conditional information is not uniform throughout sampling, but becomes significantly stronger after a critical stage characterized by a Fisher-information balance condition. Motivated by this insight, we propose a training-free dynamic CFG strategy for inference. Rather than prescribing a fixed schedule in advance, our method constructs a stochastic process from conditional and unconditional score statistics and activates guidance when the estimated guidance gain crosses a prescribed threshold. This yields an adaptive activation rule that aligns condition injection with the underlying generation dynamics, while also recovering interval-based guidance as a limiting special case. Extensive experiments across diverse architectures and tasks, demonstrate that our method is effective, robust, and broadly applicable. Beyond empirical improvements, our framework provides an interpretation of dynamic guidance methods and a principled foundation for inference-time control in conditional diffusion models.