mlsanityData to 7 Oct 2026

SoulX-FlashTalk: Real-Time Infinite Streaming of Audio-Driven Avatars via Self-Correcting Bidirectional Distillation

arXiv 2512.23379 ↗First posted 29 Dec 2025cs.CV cs.AIcited by 10

On the map: Diffusion models → Audio, speech and music

Abstract (arXiv)

Deploying massive diffusion models for real-time, infinite-duration, audio-driven avatar generation presents a significant engineering challenge, primarily due to the conflict between computational load and strict latency constraints. Existing approaches often compromise visual fidelity by enforcing strictly unidirectional attention mechanisms or reducing model capacity. To address this problem, we introduce SoulX-FlashTalk, a 14B-parameter framework optimized for high-fidelity real-time streaming. Diverging from conventional unidirectional paradigms, we use a Self-correcting Bidirectional Distillation strategy that retains bidirectional attention within video chunks. This design preserves critical spatiotemporal correlations, significantly enhancing motion coherence and visual detail. To ensure stability during infinite generation, we incorporate a Multi-step Retrospective Self-Correction Mechanism, enabling the model to autonomously recover from accumulated errors and preventing collapse. Furthermore, we engineered a full-stack inference acceleration suite incorporating hybrid sequence parallelism, Parallel VAE, and kernel-level optimizations. Extensive evaluations confirm that SoulX-FlashTalk is the first 14B-scale system to achieve a sub-second start-up latency (0.87s) while reaching a real-time throughput of 32 FPS, setting a new standard for high-fidelity interactive digital human synthesis.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Placed in Audio, speech and music, one of 14 directions in Diffusion models. Builds on 24 of its 28 references that are in the map. First posted on arXiv 29 Dec 2025.

Builds on

24 of 28 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025StableAvatar: Infinite-Length Audio-Driven Avatar Video Generation11 Aug
  2. 2024Ditto: Motion-Space Diffusion for Controllable Realtime Talking Head Synthesis29 Nov

Other references · 22

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 2022Scalable Diffusion Models with Transformerscited by 5,931
  4. 2022Classifier-Free Diffusion Guidancecited by 5,717
  5. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  6. 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,536
  7. 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,755
  8. 2023One-step Diffusion with Distribution Matching Distillationcited by 978
  9. 2023PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallelcited by 641
  10. 2025Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusioncited by 569
  11. 2024FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precisioncited by 507
  12. 2023Q-Align: Teaching LMMs for Visual Scoring via Discrete Text-Defined Levelscited by 489
10 more
  1. 2023Ring Attention with Blockwise Transformers for Near-Infinite Contextcited by 448
  2. 2023DeepSpeed Ulysses: System Optimizations for Enabling Training of Extreme Long Sequence Transformer Modelscited by 239
  3. 2024LivePortrait: Efficient Portrait Animation with Stitching and Retargeting Controlcited by 202
  4. 2025Self-Forcing++: Towards Minute-Scale High-Quality Video Generationcited by 176
  5. 2025OmniAvatar: Efficient Audio-Driven Avatar Video Generation with Adaptive Body Animationcited by 77
  6. 2025Wan-S2V: Audio-Driven Cinematic Video Generationcited by 60
  7. 2025InfiniteTalk: Audio-driven Video Generation for Sparse-Frame Video Dubbingcited by 40
  8. 2025EchoMimicV3: 1.3B Parameters are All You Need for Unified Multi-Modal and Multi-Task Human Animationcited by 28
  9. 2025SpeakerVid-5M: A Large-Scale High-Quality Dataset for Audio-Visual Dyadic Interactive Human Generationcited by 27
  10. 2025AnyTalker: Scaling Multi-Person Talking Video Generation with Interactivity Refinementcited by 19

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 10.

10 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2025Live Avatar: Streaming Real-time Audio-Driven Avatar Generation with Infinite Length
  2. 2026AvatarForcing: One-Step Streaming Talking Avatars via Local-Future Sliding-Window Denoising
  3. 2026Omni-LiveAvatar: Minute-Level Real-Time Streaming Joint Audio-Video Avatar Generation
  4. 2026Hallo-Live: Real-Time Streaming Joint Audio-Video Avatar Generation with Asynchronous Dual-Stream and Human-Centric Preference Distillation
  5. 2026Where and When to Force: Routed Forcing for Streaming Avatars
  6. 2025TalkingMachines: Real-Time Audio-Driven FaceTime-Style Video via Autoregressive Diffusion Models
  7. 2026OmniMate: Open-Ended Real-Time Streaming Audio-Visual Generation for Interactive Avatars
  8. 2025StreamAvatar: Streaming Diffusion Models for Real-Time Interactive Human Avatars