Live Avatar: Streaming Real-time Audio-Driven Avatar Generation with Infinite Length
On the map: Diffusion models → Audio, speech and music
Abstract (arXiv)
Audio-driven avatar interaction demands real-time, streaming, and infinite-length generation -- capabilities fundamentally at odds with the sequential denoising and long-horizon drift of current diffusion models. We present Live Avatar, an algorithm-system co-designed framework that addresses both challenges for a 14-billion-parameter diffusion model. On the algorithm side, a two-stage pipeline distills a pretrained bidirectional model into a causal, few-step streaming one, while a set of complementary long-horizon strategies eliminate identity drift and visual artifacts, enabling stable autoregressive generation exceeding 10000 seconds. On the system side, Timestep-forcing Pipeline Parallelism (TPP) assigns each GPU a fixed denoising timestep, converting the sequential diffusion chain into an asynchronous spatial pipeline that simultaneously boosts throughput and improves temporal consistency. Live Avatar achieves 45 FPS with a TTFF of 1.21,s on 5 H800 GPUs, and to our knowledge is the first to enable practical real-time streaming of a 14B diffusion model for infinite-length avatar generation. We further introduce GenBench, a standardized long-form benchmark, to facilitate reproducible evaluation. Our project page is at https://liveavatar.github.io/.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Audio, speech and music, one of 14 directions in Diffusion models. Builds on 50 of its 60 references that are in the map. First posted on arXiv 4 Dec 2025.
Builds on
50 of 60 references.
Closest ancestors · 8
Same direction, newest first
- 2025StableAvatar: Infinite-Length Audio-Driven Avatar Video Generation11 Aug
- 2025RAP: Real-time Audio-driven Portrait Animation with Video Diffusion Transformer7 Aug
- 2025LLIA -- Enabling Low-Latency Interactive Avatars: Real-Time Audio-Driven Portrait Video Generation with Diffusion Models6 Jun
- 2025TalkingMachines: Real-Time Audio-Driven FaceTime-Style Video via Autoregressive Diffusion Models3 Jun
- 2024CosyVoice 2: Scalable Streaming Speech Synthesis with Large Language Models13 Dec
- 2024Hallo3: Highly Dynamic and Realistic Portrait Image Animation with Video Diffusion Transformer1 Dec
- 2024Ditto: Motion-Space Diffusion for Controllable Realtime Talking Head Synthesis29 Nov
- 2024EMO: Emote Portrait Alive -- Generating Expressive Portrait Videos with Audio2Video Diffusion Model under Weak Conditions27 Feb
Other references · 42
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
- 2022Flow Matching for Generative Modelingcited by 5,426
- 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
- 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,536
- 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,755
- 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,597
- 2018Towards Accurate Generative Models of Video: A New Metric & Challengescited by 1,204
- 2023One-step Diffusion with Distribution Matching Distillationcited by 978
- 2020A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wildcited by 669
- 2024Improved Distribution Matching Distillation for Fast Image Synthesiscited by 648
- 2024Autoregressive Image Generation without Vector Quantizationcited by 644
- 2024Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusioncited by 634
30 more
- 2025Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusioncited by 569
- 2024LTX-Video: Realtime Video Latent Diffusioncited by 549
- 2023Q-Align: Teaching LMMs for Visual Scoring via Discrete Text-Defined Levelscited by 489
- 2024From Slow Bidirectional to Fast Autoregressive Video Diffusion Modelscited by 401
- 2022SadTalker: Learning Realistic 3D Motion Coefficients for Stylized Audio-Driven Single Image Talking Face Animationcited by 380
- 2024One Step Diffusion via Shortcut Modelscited by 373
- 2024Simplifying, Stabilizing and Scaling Continuous-Time Consistency Modelscited by 276
- 2025Z-Image: An Efficient Image Generation Foundation Model with Single-Stream Diffusion Transformercited by 247
- 2025LongLive: Real-time Interactive Long Video Generationcited by 237
- 2023Diff-Instruct: A Universal Approach for Transferring Knowledge From Pre-trained Diffusion Modelscited by 232
- 2025Rolling Forcing: Autoregressive Long Video Diffusion in Real Timecited by 209
- 2025Self-Forcing++: Towards Minute-Scale High-Quality Video Generationcited by 176
- 2018Looking to Listen at the Cocktail Party: A Speaker-Independent Audio-Visual Model for Speech Separationcited by 160
- 2025Large Scale Diffusion Distillation via Score-Regularized Continuous-Time Consistencycited by 81
- 2025FantasyTalking: Realistic Talking Portrait Generation via Coherent Motion Synthesiscited by 77
- 2025OmniAvatar: Efficient Audio-Driven Avatar Video Generation with Adaptive Body Animationcited by 77
- 2024EchoMimicV2: Towards Striking, Simplified, and Semi-Body Human Animationcited by 69
- 2025Wan-S2V: Audio-Driven Cinematic Video Generationcited by 60
- 2025Learning Few-Step Diffusion Models by Trajectory Distribution Matchingcited by 49
- 2025StreamDiT: Real-Time Streaming Text-to-Video Generationcited by 43
- 2024INFP: Audio-Driven Interactive Head Generation in Dyadic Conversationscited by 40
- 2025InfiniteTalk: Audio-driven Video Generation for Sparse-Frame Video Dubbingcited by 40
- 2024Diff-Instruct++: Training One-step Text-to-image Generator Model to Align with Human Preferencescited by 30
- 2025MIDAS: Multimodal Interactive Digital-humAn Synthesis via Real-time Autoregressive Video Generationcited by 18
- 2025Teller: Real-Time Streaming Audio-Driven Portrait Animation with Autoregressive Motion Generationcited by 16
- 2025ARIG: Autoregressive Interactive Head Generation for Real-time Conversationscited by 16
- 2025X-Streamer: Unified Human World Modeling with Audiovisual Interactioncited by 16
- 2024Body of Her: A Preliminary Study on End-to-End Humanoid Agentcited by 15
- 2025MirrorMe: Towards Realtime and High Fidelity Audio-Driven Halfbody Animationcited by 2
- 2025THEval. Evaluation Framework for Talking Head Video Generationcited by 1
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2026Hallo-Live: Real-Time Streaming Joint Audio-Video Avatar Generation with Asynchronous Dual-Stream and Human-Centric Preference Distillation
- 2026AvatarForcing: One-Step Streaming Talking Avatars via Local-Future Sliding-Window Denoising
- 2025SoulX-FlashTalk: Real-Time Infinite Streaming of Audio-Driven Avatars via Self-Correcting Bidirectional Distillation
- 2026Omni-LiveAvatar: Minute-Level Real-Time Streaming Joint Audio-Video Avatar Generation
- 2025StreamAvatar: Streaming Diffusion Models for Real-Time Interactive Human Avatars
- 2026OmniMate: Open-Ended Real-Time Streaming Audio-Visual Generation for Interactive Avatars
- 2025Livatar-1: Real-Time Talking Heads Generation with Tailored Flow Matching
- 2026AVTR-1: Open Stack for Real-Time Interactive Avatars