Speed by Simplicity: A Single-Stream Architecture for Fast Audio-Video Generative Foundation Model
Not on the map. Why?
Abstract (arXiv)
We present daVinci-MagiHuman, an open-source audio-video generative foundation model for human-centric generation. daVinci-MagiHuman jointly generates synchronized video and audio using a single-stream Transformer that processes text, video, and audio within a unified token sequence via self-attention only. This single-stream design avoids the complexity of multi-stream or cross-attention architectures while remaining easy to optimize with standard training and inference infrastructure. The model is particularly strong in human-centric scenarios, producing expressive facial performance, natural speech-expression coordination, realistic body motion, and precise audio-video synchronization. It supports multilingual spoken generation across Chinese (Mandarin and Cantonese), English, Japanese, Korean, German, and French. For efficient inference, we combine the single-stream backbone with model distillation, latent-space super-resolution, and a Turbo VAE decoder, enabling generation of a 5-second 256p video in 2 seconds on a single H100 GPU. In automatic evaluation, daVinci-MagiHuman achieves the highest visual quality and text alignment among leading open models, along with the lowest word error rate (14.60%) for speech intelligibility. In pairwise human evaluation, it achieves win rates of 80.0% against Ovi 1.1 and 60.9% against LTX 2.3 over 2000 comparisons. We open-source the complete model stack, including the base model, the distilled model, the super-resolution model, and the inference codebase.
Key facts. Cited by 23 papers in the map (data through 7 Oct 2026). Builds on 11 of its 18 references that are in the map. First posted on arXiv 23 Mar 2026.
Builds on
11 of 18 references.
References · 11
- 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,536
- 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,597
- 2024Improved Distribution Matching Distillation for Fast Image Synthesiscited by 648
- 2025Gated Attention for Large Language Models: Non-linearity, Sparsity, and Attention-Sink-Freecited by 266
- 2026LTX-2: Efficient Joint Audio-Visual Foundation Modelcited by 213
- 2025Ovi: Twin Backbone Cross-Modal Fusion for Audio-Video Generationcited by 76
- 2025UniVerse-1: Unified Audio-Video Generation via Stitching of Expertscited by 58
- 2025Is Noise Conditioning Necessary for Denoising Generative Models?cited by 42
- 2026MOVA: Towards Scalable and Synchronized Video-Audio Generationcited by 39
- 2025VideoScore2: Think before You Score in Generative Video Evaluationcited by 37
- 2025Turbo-VAED: Fast and Stable Transfer of Video-VAEs to Mobile Devicescited by 15
Built on by
Citing papers per year: 2026: 23.
23 papers.
Sort:
By area:
- 2026Hallo-Live: Real-Time Streaming Joint Audio-Video Avatar Generation with Asynchronous Dual-Stream and Human-Centric Preference Distillationcited by 13
- 2026LongAV-Compass: Towards Unified Evaluation of Minute-Scale Audio-Visual Generation Across T2AV, I2AV, and V2AVcited by 9
- 2026Do Joint Audio-Video Generation Models Understand Physics?cited by 8
- 2026MSAVBench: Towards Comprehensive and Reliable Evaluation of Multi-Shot Audio-Video Generationcited by 8
- 2026Script-a-Video: Deep Structured Audio-visual Captions via Factorized Streams and Relational Groundingcited by 6
- 2026Native Audio-Visual Alignment for Generationcited by 6
- 2026Artifact-Bench: Evaluating MLLMs on Detecting and Assessing the Artifacts of AI-Generated Videoscited by 4
- 2026StreamChar: Long-Horizon Streaming Character Audio-Video Generation with Decoupled Orchestrationcited by 3
- 2026VA-Judger: Reward Modeling from Human Preference Feedback for Joint Video-Audio Generationcited by 3
- 2026Ultra Flash: Scaling Real-Time Streaming Video Generation to High Resolutionscited by 1
- 2026KeyFrame-Compass: Towards Comprehensive Evaluation of Keyframe-Conditioned Video Generationcited by 1
- 2026Avatar-Forever: Decoupled Parallel Training for High-Quality Real-Time Infinite Avatarscited by 1
11 more
- 2026Omni-LiveAvatar: Minute-Level Real-Time Streaming Joint Audio-Video Avatar Generationcited by 1
- 2026Beyond Coherence: Benchmarking Professional Editing-Technique Execution in Multi-Shot Audio-Video Generationcited by 1
- 2026Salt++: Context-Aligned Post-Training for Few-Step Streaming Multimodal Generationcited by 1
- 2026OmniMate: Open-Ended Real-Time Streaming Audio-Visual Generation for Interactive Avatarscited by 0
- 2026Ripple: Real-Time Streaming Audio-Video Generation With Cross-Modal Recurrent Memorycited by 0
- 2026Vorch-Omni: Multi-Task Orchestration of Sight and Soundcited by 0
- 2026TurboT2VA: Fast Large-Scale Text-to-Video-Audio Generation via Score-Regularized Consistency Distillationcited by 0
- 2026Multi2AV-Safety: Benchmarking Safety in Multimodal-to-Audio-Video Generationcited by 0
- 2026DreamX-Creator: Democratizing Native Audio-Video Generation at 2K Resolutioncited by 0
- 2026TimeSteer: Inference-Time Speech Scheduling in Joint Audio-Visual Diffusion Modelscited by 0
- 2026Vorch-Human: Unified Multi-Task Human-Centric Generation via Long-Horizon Continuationcited by 0
- 2026Salt++: Context-Aligned Post-Training for Few-Step Streaming Multimodal Generation29 Sep 2026
- 2026Beyond Coherence: Benchmarking Professional Editing-Technique Execution in Multi-Shot Audio-Video Generation8 Sep 2026
- 2026TimeSteer: Inference-Time Speech Scheduling in Joint Audio-Visual Diffusion Models1 Sep 2026
- 2026DreamX-Creator: Democratizing Native Audio-Video Generation at 2K Resolution31 Aug 2026
- 2026Multi2AV-Safety: Benchmarking Safety in Multimodal-to-Audio-Video Generation27 Aug 2026
- 2026TurboT2VA: Fast Large-Scale Text-to-Video-Audio Generation via Score-Regularized Consistency Distillation25 Aug 2026
- 2026VA-Judger: Reward Modeling from Human Preference Feedback for Joint Video-Audio Generation19 Aug 2026
- 2026Avatar-Forever: Decoupled Parallel Training for High-Quality Real-Time Infinite Avatars12 Aug 2026
- 2026Omni-LiveAvatar: Minute-Level Real-Time Streaming Joint Audio-Video Avatar Generation7 Aug 2026
- 2026Vorch-Human: Unified Multi-Task Human-Centric Generation via Long-Horizon Continuation6 Aug 2026
- 2026Vorch-Omni: Multi-Task Orchestration of Sight and Sound6 Aug 2026
- 2026Ripple: Real-Time Streaming Audio-Video Generation With Cross-Modal Recurrent Memory29 Jul 2026
11 more
- 2026OmniMate: Open-Ended Real-Time Streaming Audio-Visual Generation for Interactive Avatars25 Jul 2026
- 2026KeyFrame-Compass: Towards Comprehensive Evaluation of Keyframe-Conditioned Video Generation15 Jul 2026
- 2026Ultra Flash: Scaling Real-Time Streaming Video Generation to High Resolutions8 Jun 2026
- 2026Native Audio-Visual Alignment for Generation28 May 2026
- 2026LongAV-Compass: Towards Unified Evaluation of Minute-Scale Audio-Visual Generation Across T2AV, I2AV, and V2AV25 May 2026
- 2026StreamChar: Long-Horizon Streaming Character Audio-Video Generation with Decoupled Orchestration25 May 2026
- 2026MSAVBench: Towards Comprehensive and Reliable Evaluation of Multi-Shot Audio-Video Generation19 May 2026
- 2026Artifact-Bench: Evaluating MLLMs on Detecting and Assessing the Artifacts of AI-Generated Videos18 May 2026
- 2026Do Joint Audio-Video Generation Models Understand Physics?8 May 2026
- 2026Hallo-Live: Real-Time Streaming Joint Audio-Video Avatar Generation with Asynchronous Dual-Stream and Human-Centric Preference Distillation26 Apr 2026
- 2026Script-a-Video: Deep Structured Audio-visual Captions via Factorized Streams and Relational Grounding13 Apr 2026
- 2026Hallo-Live: Real-Time Streaming Joint Audio-Video Avatar Generation with Asynchronous Dual-Stream and Human-Centric Preference Distillationcited by 13
- 2026Ultra Flash: Scaling Real-Time Streaming Video Generation to High Resolutionscited by 1
- 2026OmniMate: Open-Ended Real-Time Streaming Audio-Visual Generation for Interactive Avatarscited by 0
- 2026Vorch-Omni: Multi-Task Orchestration of Sight and Soundcited by 0
- 2026TimeSteer: Inference-Time Speech Scheduling in Joint Audio-Visual Diffusion Modelscited by 0
- 2026TimeSteer: Inference-Time Speech Scheduling in Joint Audio-Visual Diffusion Models1 Sep 2026
- 2026Vorch-Omni: Multi-Task Orchestration of Sight and Sound6 Aug 2026
- 2026OmniMate: Open-Ended Real-Time Streaming Audio-Visual Generation for Interactive Avatars25 Jul 2026
- 2026Ultra Flash: Scaling Real-Time Streaming Video Generation to High Resolutions8 Jun 2026
- 2026Hallo-Live: Real-Time Streaming Joint Audio-Video Avatar Generation with Asynchronous Dual-Stream and Human-Centric Preference Distillation26 Apr 2026
- 2026LongAV-Compass: Towards Unified Evaluation of Minute-Scale Audio-Visual Generation Across T2AV, I2AV, and V2AVcited by 9
- 2026Do Joint Audio-Video Generation Models Understand Physics?cited by 8
- 2026MSAVBench: Towards Comprehensive and Reliable Evaluation of Multi-Shot Audio-Video Generationcited by 8
- 2026Native Audio-Visual Alignment for Generationcited by 6
- 2026Artifact-Bench: Evaluating MLLMs on Detecting and Assessing the Artifacts of AI-Generated Videoscited by 4
- 2026StreamChar: Long-Horizon Streaming Character Audio-Video Generation with Decoupled Orchestrationcited by 3
- 2026KeyFrame-Compass: Towards Comprehensive Evaluation of Keyframe-Conditioned Video Generationcited by 1
- 2026Avatar-Forever: Decoupled Parallel Training for High-Quality Real-Time Infinite Avatarscited by 1
- 2026Omni-LiveAvatar: Minute-Level Real-Time Streaming Joint Audio-Video Avatar Generationcited by 1
- 2026Beyond Coherence: Benchmarking Professional Editing-Technique Execution in Multi-Shot Audio-Video Generationcited by 1
- 2026Salt++: Context-Aligned Post-Training for Few-Step Streaming Multimodal Generationcited by 1
- 2026Ripple: Real-Time Streaming Audio-Video Generation With Cross-Modal Recurrent Memorycited by 0
12 most cited of 16 All 16 (JSON)
- 2026Salt++: Context-Aligned Post-Training for Few-Step Streaming Multimodal Generation29 Sep 2026
- 2026Beyond Coherence: Benchmarking Professional Editing-Technique Execution in Multi-Shot Audio-Video Generation8 Sep 2026
- 2026DreamX-Creator: Democratizing Native Audio-Video Generation at 2K Resolution31 Aug 2026
- 2026Multi2AV-Safety: Benchmarking Safety in Multimodal-to-Audio-Video Generation27 Aug 2026
- 2026TurboT2VA: Fast Large-Scale Text-to-Video-Audio Generation via Score-Regularized Consistency Distillation25 Aug 2026
- 2026Avatar-Forever: Decoupled Parallel Training for High-Quality Real-Time Infinite Avatars12 Aug 2026
- 2026Omni-LiveAvatar: Minute-Level Real-Time Streaming Joint Audio-Video Avatar Generation7 Aug 2026
- 2026Vorch-Human: Unified Multi-Task Human-Centric Generation via Long-Horizon Continuation6 Aug 2026
- 2026Ripple: Real-Time Streaming Audio-Video Generation With Cross-Modal Recurrent Memory29 Jul 2026
- 2026KeyFrame-Compass: Towards Comprehensive Evaluation of Keyframe-Conditioned Video Generation15 Jul 2026
- 2026Native Audio-Visual Alignment for Generation28 May 2026
- 2026LongAV-Compass: Towards Unified Evaluation of Minute-Scale Audio-Visual Generation Across T2AV, I2AV, and V2AV25 May 2026
12 newest of 16 All 16 (JSON)