mlsanityData to 7 Oct 2026

Speed by Simplicity: A Single-Stream Architecture for Fast Audio-Video Generative Foundation Model

arXiv 2603.21986 ↗First posted 23 Mar 2026cs.CVcited by 23

Not on the map. Why?

Abstract (arXiv)

We present daVinci-MagiHuman, an open-source audio-video generative foundation model for human-centric generation. daVinci-MagiHuman jointly generates synchronized video and audio using a single-stream Transformer that processes text, video, and audio within a unified token sequence via self-attention only. This single-stream design avoids the complexity of multi-stream or cross-attention architectures while remaining easy to optimize with standard training and inference infrastructure. The model is particularly strong in human-centric scenarios, producing expressive facial performance, natural speech-expression coordination, realistic body motion, and precise audio-video synchronization. It supports multilingual spoken generation across Chinese (Mandarin and Cantonese), English, Japanese, Korean, German, and French. For efficient inference, we combine the single-stream backbone with model distillation, latent-space super-resolution, and a Turbo VAE decoder, enabling generation of a 5-second 256p video in 2 seconds on a single H100 GPU. In automatic evaluation, daVinci-MagiHuman achieves the highest visual quality and text alignment among leading open models, along with the lowest word error rate (14.60%) for speech intelligibility. In pairwise human evaluation, it achieves win rates of 80.0% against Ovi 1.1 and 60.9% against LTX 2.3 over 2000 comparisons. We open-source the complete model stack, including the base model, the distilled model, the super-resolution model, and the inference codebase.

Key facts. Cited by 23 papers in the map (data through 7 Oct 2026). Builds on 11 of its 18 references that are in the map. First posted on arXiv 23 Mar 2026.

Builds on

11 of 18 references.

References · 11

  1. 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,536
  2. 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,597
  3. 2024Improved Distribution Matching Distillation for Fast Image Synthesiscited by 648
  4. 2025Gated Attention for Large Language Models: Non-linearity, Sparsity, and Attention-Sink-Freecited by 266
  5. 2026LTX-2: Efficient Joint Audio-Visual Foundation Modelcited by 213
  6. 2025Ovi: Twin Backbone Cross-Modal Fusion for Audio-Video Generationcited by 76
  7. 2025UniVerse-1: Unified Audio-Video Generation via Stitching of Expertscited by 58
  8. 2025Is Noise Conditioning Necessary for Denoising Generative Models?cited by 42
  9. 2026MOVA: Towards Scalable and Synchronized Video-Audio Generationcited by 39
  10. 2025VideoScore2: Think before You Score in Generative Video Evaluationcited by 37
  11. 2025Turbo-VAED: Fast and Stable Transfer of Video-VAEs to Mobile Devicescited by 15

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 23.

23 papers.

Sort:

By area:

  1. 2026Hallo-Live: Real-Time Streaming Joint Audio-Video Avatar Generation with Asynchronous Dual-Stream and Human-Centric Preference Distillationcited by 13
  2. 2026LongAV-Compass: Towards Unified Evaluation of Minute-Scale Audio-Visual Generation Across T2AV, I2AV, and V2AVcited by 9
  3. 2026Do Joint Audio-Video Generation Models Understand Physics?cited by 8
  4. 2026MSAVBench: Towards Comprehensive and Reliable Evaluation of Multi-Shot Audio-Video Generationcited by 8
  5. 2026Script-a-Video: Deep Structured Audio-visual Captions via Factorized Streams and Relational Groundingcited by 6
  6. 2026Native Audio-Visual Alignment for Generationcited by 6
  7. 2026Artifact-Bench: Evaluating MLLMs on Detecting and Assessing the Artifacts of AI-Generated Videoscited by 4
  8. 2026StreamChar: Long-Horizon Streaming Character Audio-Video Generation with Decoupled Orchestrationcited by 3
  9. 2026VA-Judger: Reward Modeling from Human Preference Feedback for Joint Video-Audio Generationcited by 3
  10. 2026Ultra Flash: Scaling Real-Time Streaming Video Generation to High Resolutionscited by 1
  11. 2026KeyFrame-Compass: Towards Comprehensive Evaluation of Keyframe-Conditioned Video Generationcited by 1
  12. 2026Avatar-Forever: Decoupled Parallel Training for High-Quality Real-Time Infinite Avatarscited by 1
11 more
  1. 2026Omni-LiveAvatar: Minute-Level Real-Time Streaming Joint Audio-Video Avatar Generationcited by 1
  2. 2026Beyond Coherence: Benchmarking Professional Editing-Technique Execution in Multi-Shot Audio-Video Generationcited by 1
  3. 2026Salt++: Context-Aligned Post-Training for Few-Step Streaming Multimodal Generationcited by 1
  4. 2026OmniMate: Open-Ended Real-Time Streaming Audio-Visual Generation for Interactive Avatarscited by 0
  5. 2026Ripple: Real-Time Streaming Audio-Video Generation With Cross-Modal Recurrent Memorycited by 0
  6. 2026Vorch-Omni: Multi-Task Orchestration of Sight and Soundcited by 0
  7. 2026TurboT2VA: Fast Large-Scale Text-to-Video-Audio Generation via Score-Regularized Consistency Distillationcited by 0
  8. 2026Multi2AV-Safety: Benchmarking Safety in Multimodal-to-Audio-Video Generationcited by 0
  9. 2026DreamX-Creator: Democratizing Native Audio-Video Generation at 2K Resolutioncited by 0
  10. 2026TimeSteer: Inference-Time Speech Scheduling in Joint Audio-Visual Diffusion Modelscited by 0
  11. 2026Vorch-Human: Unified Multi-Task Human-Centric Generation via Long-Horizon Continuationcited by 0