mlsanityData to 7 Oct 2026

SoulX-FlashHead: Oracle-guided Generation of Infinite Real-time Streaming Talking Heads

arXiv 2602.07449 ↗First posted 7 Feb 2026cs.CVcited by 4

Not on the map. Why?

Abstract (arXiv)

Achieving a balance between high-fidelity visual quality and low-latency streaming remains a formidable challenge in audio-driven portrait generation. Existing large-scale models often suffer from prohibitive computational costs, while lightweight alternatives typically compromise on holistic facial representations and temporal stability. In this paper, we propose SoulX-FlashHead, a unified 1.3B-parameter framework designed for real-time, infinite-length, and high-fidelity streaming video generation. To address the instability of audio features in streaming scenarios, we introduce Streaming-Aware Spatiotemporal Pre-training equipped with a Temporal Audio Context Cache mechanism, which ensures robust feature extraction from short audio fragments. Furthermore, to mitigate the error accumulation and identity drift inherent in long-sequence autoregressive generation, we propose Oracle-Guided Bidirectional Distillation, leveraging ground-truth motion priors to provide precise physical guidance. We also present VividHead, a large-scale, high-quality dataset containing 782 hours of strictly aligned footage to support robust training. Extensive experiments demonstrate that SoulX-FlashHead achieves state-of-the-art performance on HDTF and VFHQ benchmarks. Notably, our Lite variant achieves an inference speed of 96 FPS on a single NVIDIA RTX 4090, facilitating ultra-fast interaction without sacrificing visual coherence.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 24 of its 34 references that are in the map. First posted on arXiv 7 Feb 2026.

Builds on

24 of 34 references.

References · 24

  1. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2022Scalable Diffusion Models with Transformerscited by 5,931
  4. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  5. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,551
  6. 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,536
  7. 2015FlowNet: Learning Optical Flow with Convolutional Networkscited by 1,054
  8. 2023One-step Diffusion with Distribution Matching Distillationcited by 978
  9. 2024Open-Sora: Democratizing Efficient Video Production for Allcited by 637
  10. 2024LTX-Video: Realtime Video Latent Diffusioncited by 549
  11. 2022SadTalker: Learning Realistic 3D Motion Coefficients for Stylized Audio-Driven Single Image Talking Face Animationcited by 380
  12. 2023Effective Whole-body Pose Estimation with Two-stages Distillationcited by 307
12 more
  1. 2024LivePortrait: Efficient Portrait Animation with Stitching and Retargeting Controlcited by 202
  2. 2024EchoMimic: Lifelike Audio-Driven Portrait Animations through Editable Landmark Conditionscited by 196
  3. 2024AniPortrait: Audio-Driven Synthesis of Photorealistic Portrait Animationcited by 191
  4. 2022VFHQ: A High-Quality Dataset and Benchmark for Video Face Super-Resolutioncited by 174
  5. 2018Looking to Listen at the Cocktail Party: A Speaker-Independent Audio-Visual Model for Speech Separationcited by 160
  6. 2024Hallo3: Highly Dynamic and Realistic Portrait Image Animation with Video Diffusion Transformercited by 121
  7. 2024Sonic: Shifting Focus to Global Audio Perception in Portrait Animationcited by 70
  8. 2025Wan-S2V: Audio-Driven Cinematic Video Generationcited by 60
  9. 2024Ditto: Motion-Space Diffusion for Controllable Realtime Talking Head Synthesiscited by 41
  10. 2025InfiniteTalk: Audio-driven Video Generation for Sparse-Frame Video Dubbingcited by 40
  11. 2025SpeakerVid-5M: A Large-Scale High-Quality Dataset for Audio-Visual Dyadic Interactive Human Generationcited by 27
  12. 2025AnyTalker: Scaling Multi-Person Talking Video Generation with Interactivity Refinementcited by 19

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 4.

4 papers.