MeanCache: From Instantaneous to Average Velocity for Accelerating Flow Matching Inference

AI summary

Overview

Research area: Machine learning for efficient generative model inference, specifically Flow Matching and diffusion-style sampling acceleration. Technical level: Advanced. Scope: The paper introduces MeanCache, a training-free caching framework that uses average-velocity estimates from cached Jacobian-vector products and graph-based scheduling to accelerate Flow Matching models while preserving generation quality.

What This Paper Is About

Flow Matching models generate images and videos by repeatedly evaluating a neural network along a denoising trajectory, which is slow and expensive at commercial scale. Existing caching methods reuse instantaneous features or velocities to skip computations, but under high acceleration they cause trajectory drift and error accumulation. MeanCache solves this by shifting from instantaneous to average velocity and by scheduling cache reuse through a stability-aware optimization.

Key Contributions

1. Average-velocity caching perspective: MeanCache reformulates caching in the average-velocity domain instead of the instantaneous-velocity domain, using cached Jacobian-vector products (JVP) to estimate interval average velocities and reduce local error accumulation.
2. Trajectory-stability scheduling strategy: The method scores timesteps by JVP-based stability deviation, represents possible cache transitions as a multigraph, and uses a budget-constrained peak-suppressed shortest path to decide where and how long to cache.
3. State-of-the-art acceleration on commercial models: Experiments on FLUX.1, Qwen-Image, and HunyuanVideo show 4.12x, 4.56x, and 3.59x speedups, respectively, while outperforming existing caching baselines in generation quality.
4. Training-free and architecture-agnostic: The framework requires no retraining, distillation, pruning, or architecture modification, making it practical for large-scale generative models.

Main Findings

• Average velocity is smoother than instantaneous velocity: Along the denoising trajectory, instantaneous velocities fluctuate sharply, while interval average velocities are much smoother and more stable for reconstruction.
• JVP caching reduces error but depends on context: The benefit of JVP-based correction varies with timestep, cache interval, and hyperparameter K, showing that fixed caching rules are insufficient.
• Strong quality-speed trade-off on image generation: On FLUX.1, MeanCache reaches 4.12x acceleration with ImageReward 0.993 and LPIPS 0.272, while competitors degrade sharply at similar speedups. On Qwen-Image, it reaches 4.56x acceleration with LPIPS 0.236.
• Strong quality-speed trade-off on video generation: On HunyuanVideo, MeanCache achieves 3.59x acceleration with VBench 80.08%, SSIM 0.732, and PSNR 20.464, outperforming baselines such as TeaCache, TaylorSeer, DiCache, ToCa, and DuCa.
• Scheduling reveals early timesteps matter most: The learned shortest-path patterns show that early denoising steps are critical, while later steps in the second half are more suitable for skipping. The optimal JVP span K is not fixed and depends on both budget and timestep.
• Peak suppression improves quality: The peak-suppression parameter gamma penalizes high-error edges. Ablation shows gamma=5 gives the best metrics, while gamma=1 leaves error spikes and suboptimal quality.
• Better content consistency under rare-word prompts: At 4.12x acceleration, MeanCache preserves most original content and details, whereas TaylorSeer and TeaCache show severe content drift and quality degradation.
• Baselines collapse at high acceleration: Beyond roughly 3.5x on FLUX.1 and 3.0x on HunyuanVideo, competing methods suffer blurring, detail loss, and structural distortions.

Methodology in Plain English

The researchers start from the MeanFlow identity, which connects instantaneous velocity to average velocity over a time interval. They derive a version of this identity anchored at the start of the interval, then approximate the required Jacobian-vector product using cached information from an earlier timestep. This lets them correct a current instantaneous velocity into an estimated average velocity, which is smoother and less prone to error accumulation. If no useful cached JVP is available, the method falls back to the instantaneous velocity.

Because the quality of this approximation depends on when and how long to cache, the authors build a stability map. They treat each denoising timestep as a node in a graph and each possible cache transition as a directed edge weighted by the deviation between the true average velocity and the cached estimate. Multiple cache spans create a multigraph. A budget-constrained peak-suppressed shortest path then selects a sequence of full computation steps and cache reuse steps. The budget B controls the acceleration ratio: smaller B means more speedup. The peak-suppression parameter gamma discourages putting large errors into a few edges, which protects generation quality. The entire schedule is computed offline and applied at inference without retraining.

Why This Matters

Impact on research: MeanCache broadens the design space of training-free acceleration by moving from instantaneous feature reuse to average-velocity modeling. It connects theoretical ideas such as MeanFlow and JVP identities to practical large-scale generative models, and it introduces graph-based scheduling as a principled way to place caches.

Real-world applications:

• Interactive image generation and editing on consumer GPUs or cloud services with lower latency.
• Fast text-to-video generation for storyboarding, social media, and prototyping.
• Resource-constrained deployment on edge devices, mobile platforms, or cost-sensitive APIs.
• Enterprise content pipelines for marketing, e-commerce product images, and ad creative at scale.

Industry relevance: The method reduces GPU cost and latency for commercial generative models such as FLUX.1, Qwen-Image, and HunyuanVideo without retraining. This makes high-quality generation more accessible for real-time products and large-scale serving, and it can lower energy use per generated sample.

Future Directions

• Adaptive scheduling per prompt, seed, or sample, replacing the precomputed stability map with online estimation.
• Extending the average-velocity caching idea to other modalities and architectures, including audio, 3D, multi-modal generation, and different transformer or diffusion variants.
• Combining MeanCache with distillation, quantization, or pruning for compounded acceleration and broader deployment.
• Developing tighter theory for JVP approximation error, optimal K selection, and guarantees under high acceleration ratios.
• Reducing the overhead and memory cost of JVP caching and multigraph construction for extremely large models.
• Designing better evaluation metrics for content consistency and rare-concept preservation under aggressive acceleration.

Target Audience

This paper benefits machine learning researchers and engineers working on diffusion models, Flow Matching, generative model inference optimization, and model serving. It is most useful for readers with some background in generative models and numerical methods, though the high-level ideas about average velocity and cache scheduling are accessible to practitioners focused on deploying large image and video generation systems.

Authors’ abstract

We present MeanCache, a training-free caching framework for efficient Flow Matching inference. Existing caching methods reduce redundant computation but typically rely on instantaneous velocity information (e.g., feature caching), which often leads to severe trajectory deviations and error accumulation under high acceleration ratios. MeanCache introduces an average-velocity perspective: by leveraging cached Jacobian--vector products (JVP) to construct interval average velocities from instantaneous velocities, it effectively mitigates local error accumulation. To further improve cache timing and JVP reuse stability, we develop a trajectory-stability scheduling strategy as a practical tool, employing a Peak-Suppressed Shortest Path under budget constraints to determine the schedule. Experiments on FLUX.1, Qwen-Image, and HunyuanVideo demonstrate that MeanCache achieves 4.12X and 4.56X and 3.59X acceleration, respectively, while consistently outperforming state-of-the-art caching baselines in generation quality. We believe this simple yet effective approach provides a new perspective for Flow Matching inference and will inspire further exploration of stability-driven acceleration in commercial-scale generative models.