mlsanityData to 14 Oct 2026

Omni-LiveAvatar: Minute-Level Real-Time Streaming Joint Audio-Video Avatar Generation

arXiv 2608.13602 ↗First posted 7 Aug 2026cs.MM cs.CV cs.SDcited by 1

Not on the map. Why?

Abstract (arXiv)

Joint audio-video generative models serve as foundation for immersive and interactive digital-human generation. Nevertheless, most existing models rely on bidirectional attention and multi-step denoising and can generate only short clips, making them unsuitable for real-time interaction over extended durations. We present Omni-LiveAvatar, the first framework for minute-level, real-time streaming joint audio-video avatar generation. Specifically, we propose (1) a progressive autoregressive distillation pipeline that transfers a large bidirectional joint audio-video diffusion model into a few-step autoregressive generator without auxiliary stabilization mechanisms; (2) a synchronized audio-video long-short-term memory that preserves global consistency under a bounded memory budget; and (3) a hierarchical rolling prompt planning strategy that enables coherent semantic evolution and seamless prompt transitions. Extensive experiments show that Omni-LiveAvatar generates high-quality, synchronized minute-level avatars in real time. In terms of speed, it achieves a 33× generation speedup over its teacher, LTX-2, on a single NVIDIA H200 GPU; in terms of generation quality, it outperforms accelerated baselines across visual quality, audio quality, cross-modal synchronization, and human fidelity. Our code is available at https://github.com/Aoko955/Omni-LiveAvatar.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 30 of its 36 references that are in the map. First posted on arXiv 7 Aug 2026.

Builds on

30 of 36 references.

References · 30

  1. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
  2. 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,822
  3. 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,660
  4. 2023One-step Diffusion with Distribution Matching Distillationcited by 1,019
  5. 2025Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusioncited by 625
  6. 2022UTMOS: UTokyo-SaruLab System for VoiceMOS Challenge 2022cited by 390
  7. 2025VBench-2.0: Advancing Video Generation Benchmark Suite for Intrinsic Faithfulnesscited by 315
  8. 2025Improving Video Generation with Human Feedbackcited by 284
  9. 2026LTX-2: Efficient Joint Audio-Visual Foundation Modelcited by 236
  10. 2025Rolling Forcing: Autoregressive Long Video Diffusion in Real Timecited by 234
  11. 2022MM-Diffusion: Learning Multi-Modal Diffusion Models for Joint Audio and Video Generationcited by 227
  12. 2024Hallo: Hierarchical Audio-Driven Visual Synthesis for Portrait Image Animationcited by 202
18 more
  1. 2021NISQA: A Deep CNN-Self-Attention Model for Multidimensional Speech Quality Prediction with Crowdsourced Datasetscited by 192
  2. 2021DNSMOS P.835: A Non-Intrusive Perceptual Objective Speech Quality Metric to Evaluate Noise Suppressorscited by 172
  3. 2024Hallo3: Highly Dynamic and Realistic Portrait Image Animation with Video Diffusion Transformercited by 122
  4. 2024Loopy: Taming Audio-Driven Portrait Avatar with Long-Term Motion Dependencycited by 102
  5. 2024Hallo2: Long-Duration and High-Resolution Audio-Driven Portrait Image Animationcited by 97
  6. 2025Ovi: Twin Backbone Cross-Modal Fusion for Audio-Video Generationcited by 83
  7. 2024VBench++: Comprehensive and Versatile Benchmark Suite for Video Generative Modelscited by 45
  8. 2025Spatia: Video Generation with Updatable Spatial Memorycited by 40
  9. 2025SenseFlow: Scaling Distribution Matching for Flow-based Text-to-Image Distillationcited by 32
  10. 2026Speed by Simplicity: A Single-Stream Architecture for Fast Audio-Video Generative Foundation Modelcited by 25
  11. 2025QVGen: Pushing the Limit of Quantized Video Generative Modelscited by 22
  12. 2025Teller: Real-Time Streaming Audio-Driven Portrait Animation with Autoregressive Motion Generationcited by 16
  13. 2026Salt: Self-Consistent Distribution Matching with Cache-Aware Training for Fast Video Generationcited by 13
  14. 2026Hallo-Live: Real-Time Streaming Joint Audio-Video Avatar Generation with Asynchronous Dual-Stream and Human-Centric Preference Distillationcited by 13
  15. 2025LinVideo: A Post-Training Framework towards O(n) Attention in Efficient Video Generationcited by 11
  16. 2025RemedyGS: Defend 3D Gaussian Splatting against Computation Cost Attackscited by 7
  17. 2026Flash-VAED: Plug-and-Play VAE Decoders for Efficient Video Generationcited by 7
  18. 2026Real-Time Human Frontal View Synthesis from a Single Imagecited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2026OmniMate: Open-Ended Real-Time Streaming Audio-Visual Generation for Interactive Avatars
  2. 2025Live Avatar: Streaming Real-time Audio-Driven Avatar Generation with Infinite Length
  3. 2025StreamAvatar: Streaming Diffusion Models for Real-Time Interactive Human Avatars
  4. 2025StableAvatar: Infinite-Length Audio-Driven Avatar Video Generation
  5. 2025TalkingMachines: Real-Time Audio-Driven FaceTime-Style Video via Autoregressive Diffusion Models
  6. 2025LLIA -- Enabling Low-Latency Interactive Avatars: Real-Time Audio-Driven Portrait Video Generation with Diffusion Models
  7. 2025SoulX-FlashTalk: Real-Time Infinite Streaming of Audio-Driven Avatars via Self-Correcting Bidirectional Distillation
  8. 2026EchoAvatar: Real-time Generative Avatar Animation from Audio Streams