SoulX-FlashHead: Oracle-guided Generation of Infinite Real-time Streaming Talking Heads
Not on the map. Why?
Abstract (arXiv)
Achieving a balance between high-fidelity visual quality and low-latency streaming remains a formidable challenge in audio-driven portrait generation. Existing large-scale models often suffer from prohibitive computational costs, while lightweight alternatives typically compromise on holistic facial representations and temporal stability. In this paper, we propose SoulX-FlashHead, a unified 1.3B-parameter framework designed for real-time, infinite-length, and high-fidelity streaming video generation. To address the instability of audio features in streaming scenarios, we introduce Streaming-Aware Spatiotemporal Pre-training equipped with a Temporal Audio Context Cache mechanism, which ensures robust feature extraction from short audio fragments. Furthermore, to mitigate the error accumulation and identity drift inherent in long-sequence autoregressive generation, we propose Oracle-Guided Bidirectional Distillation, leveraging ground-truth motion priors to provide precise physical guidance. We also present VividHead, a large-scale, high-quality dataset containing 782 hours of strictly aligned footage to support robust training. Extensive experiments demonstrate that SoulX-FlashHead achieves state-of-the-art performance on HDTF and VFHQ benchmarks. Notably, our Lite variant achieves an inference speed of 96 FPS on a single NVIDIA RTX 4090, facilitating ultra-fast interaction without sacrificing visual coherence.
Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 24 of its 34 references that are in the map. First posted on arXiv 7 Feb 2026.
Builds on
24 of 34 references.
References · 24
- 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
- 2022Scalable Diffusion Models with Transformerscited by 5,931
- 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
- 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,551
- 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,536
- 2015FlowNet: Learning Optical Flow with Convolutional Networkscited by 1,054
- 2023One-step Diffusion with Distribution Matching Distillationcited by 978
- 2024Open-Sora: Democratizing Efficient Video Production for Allcited by 637
- 2024LTX-Video: Realtime Video Latent Diffusioncited by 549
- 2022SadTalker: Learning Realistic 3D Motion Coefficients for Stylized Audio-Driven Single Image Talking Face Animationcited by 380
- 2023Effective Whole-body Pose Estimation with Two-stages Distillationcited by 307
12 more
- 2024LivePortrait: Efficient Portrait Animation with Stitching and Retargeting Controlcited by 202
- 2024EchoMimic: Lifelike Audio-Driven Portrait Animations through Editable Landmark Conditionscited by 196
- 2024AniPortrait: Audio-Driven Synthesis of Photorealistic Portrait Animationcited by 191
- 2022VFHQ: A High-Quality Dataset and Benchmark for Video Face Super-Resolutioncited by 174
- 2018Looking to Listen at the Cocktail Party: A Speaker-Independent Audio-Visual Model for Speech Separationcited by 160
- 2024Hallo3: Highly Dynamic and Realistic Portrait Image Animation with Video Diffusion Transformercited by 121
- 2024Sonic: Shifting Focus to Global Audio Perception in Portrait Animationcited by 70
- 2025Wan-S2V: Audio-Driven Cinematic Video Generationcited by 60
- 2024Ditto: Motion-Space Diffusion for Controllable Realtime Talking Head Synthesiscited by 41
- 2025InfiniteTalk: Audio-driven Video Generation for Sparse-Frame Video Dubbingcited by 40
- 2025SpeakerVid-5M: A Large-Scale High-Quality Dataset for Audio-Visual Dyadic Interactive Human Generationcited by 27
- 2025AnyTalker: Scaling Multi-Person Talking Video Generation with Interactivity Refinementcited by 19
Built on by
Citing papers per year: 2026: 4.
4 papers.
- 2026StreamChar: Long-Horizon Streaming Character Audio-Video Generation with Decoupled Orchestrationcited by 3
- 2026LeapTalk: Breaking the Latency-Quality Trade-off in Talking Head Generationcited by 0
- 2026AVTR-1: Open Stack for Real-Time Interactive Avatarscited by 0
- 2026SWT: Self-Supervised Video Object Segmentation via Sliding, Wavelet and Transportationcited by 0
- 2026SWT: Self-Supervised Video Object Segmentation via Sliding, Wavelet and Transportation22 Sep 2026
- 2026AVTR-1: Open Stack for Real-Time Interactive Avatars19 Sep 2026
- 2026LeapTalk: Breaking the Latency-Quality Trade-off in Talking Head Generation29 Jul 2026
- 2026StreamChar: Long-Horizon Streaming Character Audio-Video Generation with Decoupled Orchestration25 May 2026