SoulX-FlashTalk: Real-Time Infinite Streaming of Audio-Driven Avatars via Self-Correcting Bidirectional Distillation
On the map: Diffusion models → Audio, speech and music
Abstract (arXiv)
Deploying massive diffusion models for real-time, infinite-duration, audio-driven avatar generation presents a significant engineering challenge, primarily due to the conflict between computational load and strict latency constraints. Existing approaches often compromise visual fidelity by enforcing strictly unidirectional attention mechanisms or reducing model capacity. To address this problem, we introduce SoulX-FlashTalk, a 14B-parameter framework optimized for high-fidelity real-time streaming. Diverging from conventional unidirectional paradigms, we use a Self-correcting Bidirectional Distillation strategy that retains bidirectional attention within video chunks. This design preserves critical spatiotemporal correlations, significantly enhancing motion coherence and visual detail. To ensure stability during infinite generation, we incorporate a Multi-step Retrospective Self-Correction Mechanism, enabling the model to autonomously recover from accumulated errors and preventing collapse. Furthermore, we engineered a full-stack inference acceleration suite incorporating hybrid sequence parallelism, Parallel VAE, and kernel-level optimizations. Extensive evaluations confirm that SoulX-FlashTalk is the first 14B-scale system to achieve a sub-second start-up latency (0.87s) while reaching a real-time throughput of 32 FPS, setting a new standard for high-fidelity interactive digital human synthesis.
Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Placed in Audio, speech and music, one of 14 directions in Diffusion models. Builds on 24 of its 28 references that are in the map. First posted on arXiv 29 Dec 2025.
Builds on
24 of 28 references.
Closest ancestors · 2
Same direction, newest first
- 2025StableAvatar: Infinite-Length Audio-Driven Avatar Video Generation11 Aug
- 2024Ditto: Motion-Space Diffusion for Controllable Realtime Talking Head Synthesis29 Nov
Other references · 22
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
- 2022Scalable Diffusion Models with Transformerscited by 5,931
- 2022Classifier-Free Diffusion Guidancecited by 5,717
- 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
- 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,536
- 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,755
- 2023One-step Diffusion with Distribution Matching Distillationcited by 978
- 2023PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallelcited by 641
- 2025Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusioncited by 569
- 2024FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precisioncited by 507
- 2023Q-Align: Teaching LMMs for Visual Scoring via Discrete Text-Defined Levelscited by 489
10 more
- 2023Ring Attention with Blockwise Transformers for Near-Infinite Contextcited by 448
- 2023DeepSpeed Ulysses: System Optimizations for Enabling Training of Extreme Long Sequence Transformer Modelscited by 239
- 2024LivePortrait: Efficient Portrait Animation with Stitching and Retargeting Controlcited by 202
- 2025Self-Forcing++: Towards Minute-Scale High-Quality Video Generationcited by 176
- 2025OmniAvatar: Efficient Audio-Driven Avatar Video Generation with Adaptive Body Animationcited by 77
- 2025Wan-S2V: Audio-Driven Cinematic Video Generationcited by 60
- 2025InfiniteTalk: Audio-driven Video Generation for Sparse-Frame Video Dubbingcited by 40
- 2025EchoMimicV3: 1.3B Parameters are All You Need for Unified Multi-Modal and Multi-Task Human Animationcited by 28
- 2025SpeakerVid-5M: A Large-Scale High-Quality Dataset for Audio-Visual Dyadic Interactive Human Generationcited by 27
- 2025AnyTalker: Scaling Multi-Person Talking Video Generation with Interactivity Refinementcited by 19
Built on by
Citing papers per year: 2026: 10.
10 papers.
- 2026Video Generation Models as World Models: Efficient Paradigms, Architectures and Algorithmscited by 5
- 2026LongCat-Video-Avatar 1.5 Technical Reportcited by 5
- 2026Vidu S2: Real-Time Interactive, Editable, and Spatial Video Generationcited by 4
- 2026StreamChar: Long-Horizon Streaming Character Audio-Video Generation with Decoupled Orchestrationcited by 3
- 2026TurboTalk: Progressive Distillation for One-Step Audio-Driven Talking Avatar Generationcited by 1
- 2026Avatar-Forever: Decoupled Parallel Training for High-Quality Real-Time Infinite Avatarscited by 1
- 2026DynaForcing: Overcoming Dynamic Collapse in Self-Forcing Distillation for Streaming Avatar Generationcited by 1
- 2026EMOSH: Expressive Motion and Shape Disentanglement for Human Animationcited by 0
- 2026Vorch-Streamer: Extending Human Audio-Visual Generation to Real-Time Long-Form Streamingcited by 0
- 2026SWT: Self-Supervised Video Object Segmentation via Sliding, Wavelet and Transportationcited by 0
- 2026SWT: Self-Supervised Video Object Segmentation via Sliding, Wavelet and Transportation22 Sep 2026
- 2026Vidu S2: Real-Time Interactive, Editable, and Spatial Video Generation10 Sep 2026
- 2026DynaForcing: Overcoming Dynamic Collapse in Self-Forcing Distillation for Streaming Avatar Generation18 Aug 2026
- 2026Avatar-Forever: Decoupled Parallel Training for High-Quality Real-Time Infinite Avatars12 Aug 2026
- 2026Vorch-Streamer: Extending Human Audio-Visual Generation to Real-Time Long-Form Streaming6 Aug 2026
- 2026EMOSH: Expressive Motion and Shape Disentanglement for Human Animation26 Jun 2026
- 2026LongCat-Video-Avatar 1.5 Technical Report26 May 2026
- 2026StreamChar: Long-Horizon Streaming Character Audio-Video Generation with Decoupled Orchestration25 May 2026
- 2026TurboTalk: Progressive Distillation for One-Step Audio-Driven Talking Avatar Generation16 Apr 2026
- 2026Video Generation Models as World Models: Efficient Paradigms, Architectures and Algorithms30 Mar 2026
Similar, not cited
Close in wording, no citation either way
- 2025Live Avatar: Streaming Real-time Audio-Driven Avatar Generation with Infinite Length
- 2026AvatarForcing: One-Step Streaming Talking Avatars via Local-Future Sliding-Window Denoising
- 2026Omni-LiveAvatar: Minute-Level Real-Time Streaming Joint Audio-Video Avatar Generation
- 2026Hallo-Live: Real-Time Streaming Joint Audio-Video Avatar Generation with Asynchronous Dual-Stream and Human-Centric Preference Distillation
- 2026Where and When to Force: Routed Forcing for Streaming Avatars
- 2025TalkingMachines: Real-Time Audio-Driven FaceTime-Style Video via Autoregressive Diffusion Models
- 2026OmniMate: Open-Ended Real-Time Streaming Audio-Visual Generation for Interactive Avatars
- 2025StreamAvatar: Streaming Diffusion Models for Real-Time Interactive Human Avatars