# Lightning Fast Caching-based Parallel Denoising Prediction for Accelerating Talking Head Generation

arXiv: https://arxiv.org/abs/2509.00052 · HTML: https://mlsanity.com/paper/2509.00052/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 23 of its 36 references that are in the map.
- First posted on arXiv 25 Aug 2025.

## Builds on (most cited first, 23 of 23)

- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps](https://mlsanity.com/paper/2206.00927/) (2022)
- [AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuning](https://mlsanity.com/paper/2307.04725/) (2023)
- [Towards Accurate Generative Models of Video: A New Metric & Challenges](https://mlsanity.com/paper/1812.01717/) (2018)
- [On Distillation of Guided Diffusion Models](https://mlsanity.com/paper/2210.03142/) (2022)
- [A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild](https://mlsanity.com/paper/2008.10010/) (2020)
- [Accurate 3D Face Reconstruction with Weakly-Supervised Learning: From Single Image to Image Set](https://mlsanity.com/paper/1903.08527/) (2019)
- [SadTalker: Learning Realistic 3D Motion Coefficients for Stylized Audio-Driven Single Image Talking Face Animation](https://mlsanity.com/paper/2211.12194/) (2022)
- [Hallo: Hierarchical Audio-Driven Visual Synthesis for Portrait Image Animation](https://mlsanity.com/paper/2406.08801/) (2024)
- [EchoMimic: Lifelike Audio-Driven Portrait Animations through Editable Landmark Conditions](https://mlsanity.com/paper/2407.08136/) (2024)
- [Audio2Head: Audio-driven One-shot Talking-head Generation with Natural Head Motion](https://mlsanity.com/paper/2107.09293/) (2021)
- [One-shot Talking Face Generation from Single-speaker Audio-Visual Correlation Learning](https://mlsanity.com/paper/2112.02749/) (2021)
- [DiTCtrl: Exploring Attention Control in Multi-Modal Diffusion Transformer for Tuning-Free Multi-Prompt Longer Video Generation](https://mlsanity.com/paper/2412.18597/) (2024)
- [AniTalker: Animate Vivid and Diverse Talking Faces through Identity-Decoupled Facial Motion Encoding](https://mlsanity.com/paper/2405.03121/) (2024)
- [KV-Edit: Training-Free Image Editing for Precise Background Preservation](https://mlsanity.com/paper/2502.17363/) (2025)
- [Diffusion Model-Based Video Editing: A Survey](https://mlsanity.com/paper/2407.07111/) (2024)
- [Ditto: Motion-Space Diffusion for Controllable Realtime Talking Head Synthesis](https://mlsanity.com/paper/2411.19509/) (2024)
- [MEMO: Memory-Guided Diffusion for Expressive Talking Video Generation](https://mlsanity.com/paper/2412.04448/) (2024)
- [VORTA: Efficient Video Diffusion via Routing Sparse Attention](https://mlsanity.com/paper/2505.18809/) (2025)
- [T2V-OptJail: Discrete Prompt Optimization for Text-to-Video Jailbreak Attacks](https://mlsanity.com/paper/2505.06679/) (2025)
- [CharaConsist: Fine-Grained Consistent Character Generation](https://mlsanity.com/paper/2507.11533/) (2025)
- [AsymRnR: Video Diffusion Transformers Acceleration with Asymmetric Reduction and Restoration](https://mlsanity.com/paper/2412.11706/) (2024)
- [SPAgent: Adaptive Task Decomposition and Model Selection for General Video Generation and Editing](https://mlsanity.com/paper/2411.18983/) (2024)

## Abstract (arXiv)

Diffusion-based talking head models generate high-quality, photorealistic videos but suffer from slow inference, limiting practical applications. Existing acceleration methods for general diffusion models fail to exploit the temporal and spatial redundancies unique to talking head generation. In this paper, we propose a task-specific framework addressing these inefficiencies through two key innovations. First, we introduce Lightning-fast Caching-based Parallel denoising prediction (LightningCP), caching static features to bypass most model layers in inference time. We also enable parallel prediction using cached features and estimated noisy latents as inputs, efficiently bypassing sequential sampling. Second, we propose Decoupled Foreground Attention (DFA) to further accelerate attention computations, exploiting the spatial decoupling in talking head videos to restrict attention to dynamic foreground regions. Additionally, we remove reference features in certain layers to bring extra speedup. Extensive experiments demonstrate that our framework significantly improves inference speed while preserving video quality.