InteractiveAvatar: Real-Time Streaming Video Generation for Consistent and Intent-Aware Avatars
Not on the map. Why?
Abstract (arXiv)
Recent diffusion-based models have enabled realistic audio-driven avatar generation in real-time streaming. However, existing approaches struggle to maintain visual temporal consistency and fail to explicitly perceive user intent in complex interactive streaming scenarios. To address these challenges, we propose InteractiveAvatar, a real-time infinite-streaming video generation framework that supports visually consistent avatar video generation and intent-aware interactions. With autoregressive distillation, InteractiveAvatar achieves real-time str-eaming generation of human avatars over arbitrarily long durations. For visual consistency, we introduce a Long-Short Visual Memory (LSVM) mechanism that flexibly compresses historical visual information into compact tokens, preserving both short-range coherence and long-term consistency. To generate avatars with speeches and actions aligned with user intent, we propose a Reasoning-Reaction Module (RRM), which incorporates a State-Cycling strategy and a Cache-Switching mechanism. Extensive experimental results over diverse scenarios demonstrate that our method achieves state-of-the-art visual consistency in long-duration generation, while enabling complex user-avatar interaction in real time.
Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 40 of its 49 references that are in the map. First posted on arXiv 22 Jun 2026.
Builds on
40 of 49 references.
References · 40
- 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
- 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,708
- 2018Towards Accurate Generative Models of Video: A New Metric & Challengescited by 1,234
- 2025SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Featurescited by 1,188
- 2018VoxCeleb2: Deep Speaker Recognitioncited by 875
- 2020A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wildcited by 672
- 2023PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallelcited by 660
- 2025Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusioncited by 625
- 2023Q-Align: Teaching LMMs for Visual Scoring via Discrete Text-Defined Levelscited by 503
- 2025Qwen3-Omni Technical Reportcited by 497
- 2024From Slow Bidirectional to Fast Autoregressive Video Diffusion Modelscited by 433
- 2022SadTalker: Learning Realistic 3D Motion Coefficients for Stylized Audio-Driven Single Image Talking Face Animationcited by 382
28 more
- 2020MakeItTalk: Speaker-Aware Talking-Head Animationcited by 312
- 2025LongLive: Real-time Interactive Long Video Generationcited by 265
- 2024EMO: Emote Portrait Alive -- Generating Expressive Portrait Videos with Audio2Video Diffusion Model under Weak Conditionscited by 235
- 2024EchoMimic: Lifelike Audio-Driven Portrait Animations through Editable Landmark Conditionscited by 196
- 2025Self-Forcing++: Towards Minute-Scale High-Quality Video Generationcited by 192
- 2019Learning Individual Styles of Conversational Gesturecited by 191
- 2024VASA-1: Lifelike Audio-Driven Talking Faces Generated in Real Timecited by 187
- 2022VFHQ: A High-Quality Dataset and Benchmark for Video Face Super-Resolutioncited by 176
- 2022Learning to Listen: Modeling Non-Deterministic Dyadic Facial Motioncited by 126
- 2024Hallo3: Highly Dynamic and Realistic Portrait Image Animation with Video Diffusion Transformercited by 122
- 2023CelebV-Text: A Large-Scale Facial Text-Video Datasetcited by 104
- 2025OmniAvatar: Efficient Audio-Driven Avatar Video Generation with Adaptive Body Animationcited by 83
- 2024OpenHumanVid: A Large-Scale High-Quality Dataset for Enhancing Human-Centric Video Generationcited by 77
- 2025FantasyTalking: Realistic Talking Portrait Generation via Coherent Motion Synthesiscited by 77
- 2025HunyuanVideo-Avatar: High-Fidelity Audio-Driven Human Animation for Multiple Characterscited by 69
- 2025Let Them Talk: Audio-Driven Multi-Person Conversational Video Generationcited by 66
- 2025AI Flow: Perspectives, Scenarios, and Approachescited by 44
- 2025StableAvatar: Infinite-Length Audio-Driven Avatar Video Generationcited by 44
- 2024VideoCLIP-XL: Advancing Long Description Understanding for Video CLIP Modelscited by 41
- 2022Positive-incentive Noisecited by 39
- 2023Variational Positive-incentive Noise: How Noise Benefits Modelscited by 38
- 2025EchoMimicV3: 1.3B Parameters are All You Need for Unified Multi-Modal and Multi-Task Human Animationcited by 29
- 2025TalkingMachines: Real-Time Audio-Driven FaceTime-Style Video via Autoregressive Diffusion Modelscited by 26
- 2025StreamAvatar: Streaming Diffusion Models for Real-Time Interactive Human Avatarscited by 25
- 2025X-Streamer: Unified Human World Modeling with Audiovisual Interactioncited by 18
- 2026FlowAct-R1: Towards Interactive Humanoid Video Generationcited by 11
- 2026Unison: Harmonizing Motion, Speech, and Sound for Human-Centric Audio-Video Generationcited by 6
- 2024Data Augmentation in Human-Centric Visioncited by 2
Built on by
Citing papers per year: 2026: 5.
5 papers.
- 2026DART: Difficulty-Adaptive Routing for Zero-Shot Video Temporal Groundingcited by 1
- 2026OmniMate: Open-Ended Real-Time Streaming Audio-Visual Generation for Interactive Avatarscited by 0
- 2026StreamTalk: Streaming Co-Speech Gesture Generation with Key-Pose Anchoringcited by 0
- 2026Towards Expressive and Faithful Audio-to-Image Generation: A Unified Multimodal Dataset and Synthesis Frameworkcited by 0
- 2026Human-Centric Intelligence in the Era of Foundation Models: A Surveycited by 0
- 2026Human-Centric Intelligence in the Era of Foundation Models: A Survey18 Aug 2026
- 2026Towards Expressive and Faithful Audio-to-Image Generation: A Unified Multimodal Dataset and Synthesis Framework10 Aug 2026
- 2026StreamTalk: Streaming Co-Speech Gesture Generation with Key-Pose Anchoring3 Aug 2026
- 2026OmniMate: Open-Ended Real-Time Streaming Audio-Visual Generation for Interactive Avatars25 Jul 2026
- 2026DART: Difficulty-Adaptive Routing for Zero-Shot Video Temporal Grounding1 Jul 2026