LeapTalk: Breaking the Latency-Quality Trade-off in Talking Head Generation
Not on the map. Why?
Abstract (arXiv)
Long-form and real-time talking-head generation remains challenging due to a latency-quality trade-off: inefficient multi-step diffusion prohibits streaming generation, whereas real-time autoregressive approaches suffer from error accumulation and identity drift. To address this drawback, we propose LeapTalk, a novel framework that achieves stable and real-time talking-head generation with a single forward step, scaling to arbitrarily long videos. At the heart of our approach lies a single-step bridge distillation scheme. On the one hand, departing from the conventional noise-to-data paradigm, we introduce a data-to-data transport formulation based on a Brownian bridge. Anchored by a persistent reference, this strategy effectively mitigates identity drift and enhances long-term temporal stability. On the other hand, to enable smooth knowledge transfer from a pre-trained diffusion teacher to the student bridge model, we explore a heterogeneous distillation framework with an SNR-aligned time transformation Phi(τ), which bridges the functional discrepancy between the two models. Moreover, we propose an audio-driven classifier-free guidance mechanism to maintain fine-grained lip synchronization under extreme step reduction. Extensive experiments demonstrate that our method achieves high-fidelity and temporally consistent video generation with only 1 step at up to 200 FPS, significantly outperforming existing approaches in both efficiency and stability. Project Page: https://zhangrongxiang.github.io/leaptalk-page/
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 30 of its 43 references that are in the map. First posted on arXiv 29 Jul 2026.
Builds on
30 of 43 references.
References · 30
- 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
- 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
- 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,700
- 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,708
- 2023One-step Diffusion with Distribution Matching Distillationcited by 1,019
- 2024Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusioncited by 681
- 2025Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusioncited by 625
- 2021Diffusion Schr"odinger Bridge with Applications to Score-Based Generative Modelingcited by 602
- 2022CelebV-HQ: A Large-Scale Video Facial Attributes Datasetcited by 250
- 2024LivePortrait: Efficient Portrait Animation with Stitching and Retargeting Controlcited by 206
- 2023FreeNoise: Tuning-Free Longer Video Diffusion via Noise Reschedulingcited by 180
- 2024FIFO-Diffusion: Generating Infinite Videos from Text without Trainingcited by 137
18 more
- 2023DiffTalk: Crafting Diffusion Models for Generalized Audio-Driven Portraits Animationcited by 136
- 2024Hallo3: Highly Dynamic and Realistic Portrait Image Animation with Video Diffusion Transformercited by 122
- 2020HOPE-Net: A Graph-based Model for Hand-Object Pose Estimationcited by 112
- 2025OmniAvatar: Efficient Audio-Driven Avatar Video Generation with Adaptive Body Animationcited by 83
- 2025FantasyTalking: Realistic Talking Portrait Generation via Coherent Motion Synthesiscited by 77
- 2025Stable Video Infinity: Infinite-Length Video Generation with Error Recyclingcited by 73
- 2025Wan-S2V: Audio-Driven Cinematic Video Generationcited by 61
- 2025End-to-End Training for Autoregressive Video Diffusion via Self-Resamplingcited by 52
- 2025StableAvatar: Infinite-Length Audio-Driven Avatar Video Generationcited by 44
- 2025InfiniteTalk: Audio-driven Video Generation for Sparse-Frame Video Dubbingcited by 41
- 2025EchoMimicV3: 1.3B Parameters are All You Need for Unified Multi-Modal and Multi-Task Human Animationcited by 29
- 2025AnyTalker: Scaling Multi-Person Talking Video Generation with Interactivity Refinementcited by 20
- 2024FrameBridge: Improving Image-to-Video Generation with Bridge Modelscited by 19
- 2025Knot Forcing: Taming Autoregressive Video Diffusion Models for Real-time Infinite Interactive Portrait Animationcited by 11
- 2025LiveTalk: Real-Time Multimodal Interactive Video Diffusion via Improved On-Policy Distillationcited by 9
- 2025REST: Diffusion-based Real-time End-to-end Streaming Talking Head Generation via ID-Context Caching and Asynchronous Streaming Distillationcited by 4
- 2026SoulX-FlashHead: Oracle-guided Generation of Infinite Real-time Streaming Talking Headscited by 4
- 2026TempoSyncDiff: Distilled Temporally-Consistent Diffusion for Low-Latency Audio-Driven Talking Head Generationcited by 1
Built on by
Not cited yet.