# Forecast then Calibrate: Feature Caching as ODE for Efficient Diffusion Transformers

arXiv: https://arxiv.org/abs/2508.16211 · HTML: https://mlsanity.com/paper/2508.16211/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models.
- Builds on 19 of its 32 references that are in the map.
- First posted on arXiv 22 Aug 2025.

## Placed in

- Diffusion models / [Diffusion foundations and sampling](https://mlsanity.com/diffusion/diffusion-foundations-and-sampling/)

## Builds on (most cited first, 19 of 19)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow](https://mlsanity.com/paper/2209.03003/) (2022)
- [CLIPScore: A Reference-free Evaluation Metric for Image Captioning](https://mlsanity.com/paper/2104.08718/) (2021)
- [DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps](https://mlsanity.com/paper/2206.00927/) (2022)
- [VBench: Comprehensive Benchmark Suite for Video Generative Models](https://mlsanity.com/paper/2311.17982/) (2023)
- [DeepCache: Accelerating Diffusion Models for Free](https://mlsanity.com/paper/2312.00858/) (2023)
- [Token Merging for Fast Stable Diffusion](https://mlsanity.com/paper/2303.17604/) (2023)
- [DPM-Solver-v3: Improved Diffusion ODE Solver with Empirical Model Statistics](https://mlsanity.com/paper/2310.13268/) (2023)
- [Cache Me if You Can: Accelerating Diffusion Models through Block Caching](https://mlsanity.com/paper/2312.03209/) (2023)
- [Real-Time Video Generation with Pyramid Attention Broadcast](https://mlsanity.com/paper/2408.12588/) (2024)
- [From Reusing to Forecasting: Accelerating Diffusion Models with TaylorSeers](https://mlsanity.com/paper/2503.06923/) (2025)
- [DiTFastAttn: Attention Compression for Diffusion Transformer Models](https://mlsanity.com/paper/2406.08552/) (2024)
- [Hunyuan-Large: An Open-Source MoE Model with 52 Billion Activated Parameters by Tencent](https://mlsanity.com/paper/2411.02265/) (2024)
- [DiP-GO: A Diffusion Pruner via Few-step Gradient Optimization](https://mlsanity.com/paper/2410.16942/) (2024)
- [Ditto: Accelerating Diffusion Model via Temporal Value Similarity](https://mlsanity.com/paper/2501.11211/) (2025)

## Built on by (most cited first, 26 of 26)

- [FreqCa: Accelerating Diffusion Models via Frequency-Aware Caching](https://mlsanity.com/paper/2510.08669/) (2025)
- [A Survey on Cache Methods in Diffusion Models: Toward Efficient Multi-Modal Generation](https://mlsanity.com/paper/2510.19755/) (2025)
- [LiveEdit: Towards Real-Time Diffusion-Based Streaming Video Editing](https://mlsanity.com/paper/2606.26740/) (2026)
- [Efficient Video Diffusion Models: Advancements and Challenges](https://mlsanity.com/paper/2604.15911/) (2026)
- [From Sketch to Fresco: Efficient Diffusion Transformer with Progressive Resolution](https://mlsanity.com/paper/2601.07462/) (2026)
- [FlashPortrait: 6x Faster Infinite Portrait Animation with Adaptive Latent Prediction](https://mlsanity.com/paper/2512.16900/) (2025)
- [LESA: Learnable Stage-Aware Predictors for Diffusion Model Acceleration](https://mlsanity.com/paper/2602.20497/) (2026)
- [Forecast the Principal, Stabilize the Residual: Subspace-Aware Feature Caching for Efficient Diffusion Transformers](https://mlsanity.com/paper/2601.07396/) (2026)
- [TAP: A Token-Adaptive Predictor Framework for Training-Free Diffusion Acceleration](https://mlsanity.com/paper/2603.03792/) (2026)
- [ResilPhase: Plug-and-Play Phase Mapping and Noise-Resilient Macro-Trajectory Extrapolation for Diffusion Acceleration](https://mlsanity.com/paper/2606.26769/) (2026)
- [Beyond Fixed Formulas: Data-Driven Linear Predictor for Efficient Diffusion Models](https://mlsanity.com/paper/2604.26365/) (2026)
- [SpecEdit: Training-Free Acceleration for Diffusion based Image Editing via Semantic Locking](https://mlsanity.com/paper/2605.02152/) (2026)
- [BeCARE: Budgeted Cache Refresh for Diffusion Transformer Acceleration](https://mlsanity.com/paper/2605.27075/) (2026)
- [Disagree to Accelerate: Closing the Loop on Diffusion Feature Forecasts](https://mlsanity.com/paper/2608.01740/) (2026)
- [MSEditor: Toward Consistent Multi-Shot Video Editing](https://mlsanity.com/paper/2608.17559/) (2026)
- [LinCa: Accelerating Diffusion Models via Learnable Decomposed Feature Caching](https://mlsanity.com/paper/2608.17973/) (2026)
- [Accelerating Masked Image Generation by Learning Controlled Latent Dynamics](https://mlsanity.com/paper/2602.23996/) (2026)
- [Dynamic Video Generation: Shaping Video Generation Across Time and Space](https://mlsanity.com/paper/2605.21042/) (2026)
- [MagicPrompt: Ultra-Lightweight Prompt Tuning for Video Generation](https://mlsanity.com/paper/2607.14595/) (2026)
- [BRACE: Taming Sharp Irregularities via Barycentric Rational Forecasting for Fast Diffusion Transformers Inference](https://mlsanity.com/paper/2608.07572/) (2026)
- [Accelerating Diffusion Transformers with Gaussian Process Rectified Feature Cache](https://mlsanity.com/paper/2609.05981/) (2026)
- [RA-CFGCache: From Branch-Level Criteria to Guided-Risk Control under Classifier-Free Guidance](https://mlsanity.com/paper/2609.36433/) (2026)
- [The Golden Path Hypothesis: Reusable Schedules in Diffusion Caching](https://mlsanity.com/paper/2609.39343/) (2026)
- [Hybrid-Basis Feature Forecasting for Diffusion Sampling Acceleration](https://mlsanity.com/paper/2610.05254/) (2026)
- [TGuide: Faster Autoregressive Video Diffusion via Historical Trajectory Guidance](https://mlsanity.com/paper/2610.14657/) (2026)
- [Tikhonov-Stabilized Bezier Representation Forecasting for Training-free Diffusion Acceleration](https://mlsanity.com/paper/2610.16448/) (2026)

## Abstract (arXiv)

Diffusion Transformers (DiTs) have demonstrated exceptional performance in high-fidelity image and video generation. To reduce their substantial computational costs, feature caching techniques have been proposed to accelerate inference by reusing hidden representations from previous timesteps. However, current methods often struggle to maintain generation quality at high acceleration ratios, where prediction errors increase sharply due to the inherent instability of long-step forecasting. In this work, we adopt an ordinary differential equation (ODE) perspective on the hidden-feature sequence, modeling layer representations along the trajectory as a feature-ODE. We attribute the degradation of existing caching strategies to their inability to robustly integrate historical features under large skipping intervals. To address this, we propose FoCa (Forecast-then-Calibrate), which treats feature caching as a feature-ODE solving problem. Extensive experiments on image synthesis, video generation, and super-resolution tasks demonstrate the effectiveness of FoCa, especially under aggressive acceleration. Without additional training, FoCa achieves near-lossless speedups of 5.50 times on FLUX, 6.45 times on HunyuanVideo, 3.17 times on Inf-DiT, and maintains high quality with a 4.53 times speedup on DiT.