MOVA: Towards Scalable and Synchronized Video-Audio Generation
Not on the map. Why?
Abstract (arXiv)
Audio is indispensable for real-world video, yet generation models have largely overlooked audio components. Current approaches to producing audio-visual content often rely on cascaded pipelines, which increase cost, accumulate errors, and degrade overall quality. While systems such as Veo 3 and Sora 2 emphasize the value of simultaneous generation, joint multimodal modeling introduces unique challenges in architecture, data, and training. Moreover, the closed-source nature of existing systems limits progress in the field. In this work, we introduce MOVA (MOSS Video and Audio), an open-source model capable of generating high-quality, synchronized audio-visual content, including realistic lip-synced speech, environment-aware sound effects, and content-aligned music. MOVA employs a Mixture-of-Experts (MoE) architecture, with a total of 32B parameters, of which 18B are active during inference. It supports IT2VA (Image-Text to Video-Audio) generation task. By releasing the model weights and code, we aim to advance research and foster a vibrant community of creators. The released codebase features comprehensive support for efficient inference, LoRA fine-tuning, and prompt enhancement.
Key facts. Cited by 39 papers in the map (data through 7 Oct 2026). Builds on 64 of its 75 references that are in the map. First posted on arXiv 9 Feb 2026.
Builds on
64 of 75 references.
References · 64
- 2020Language Models are Few-Shot Learnerscited by 32,199
- 2020Scaling Laws for Neural Language Modelscited by 6,411
- 2022Scalable Diffusion Models with Transformerscited by 5,931
- 2022Flow Matching for Generative Modelingcited by 5,426
- 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
- 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
- 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
- 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,550
- 2016Improved Techniques for Training GANscited by 2,938
- 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,591
- 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,551
- 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,597
38 more
- 2025gpt-oss-120b & gpt-oss-20b Model Cardcited by 1,258
- 2023ImageBind: One Embedding Space To Bind Them Allcited by 1,162
- 2025OpenAI GPT-5 System Cardcited by 910
- 2024WorldSimBench: Towards Video Generation Models as World Simulatorscited by 867
- 2022Large-scale Contrastive Language-Audio Pretraining with Feature Fusion and Keyword-to-Caption Augmentationcited by 707
- 2020VGGSound: A Large-scale Audio-Visual Datasetcited by 641
- 2023PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallelcited by 641
- 2024Open-Sora: Democratizing Efficient Video Production for Allcited by 637
- 2023AudioLDM: Text-to-Audio Generation with Latent Diffusion Modelscited by 592
- 2019PANNs: Large-Scale Pretrained Audio Neural Networks for Audio Pattern Recognitioncited by 563
- 2024LTX-Video: Realtime Video Latent Diffusioncited by 549
- 2025Qwen3-Omni Technical Reportcited by 451
- 2024CameraCtrl: Enabling Camera Control for Text-to-Video Generationcited by 448
- 2023High-Fidelity Audio Compression with Improved RVQGANcited by 423
- 2023AudioLDM 2: Learning Holistic Audio Generation with Self-supervised Pretrainingcited by 343
- 2024OpenVid-1M: A Large-Scale High-Quality Dataset for Text-to-video Generationcited by 332
- 2022Exploring Video Quality Assessment on User Generated Contents from Aesthetic and Technical Perspectivescited by 314
- 2023WavCaps: A ChatGPT-Assisted Weakly-Labelled Audio Captioning Dataset for Audio-Language Multimodal Researchcited by 266
- 2025LongLive: Real-time Interactive Long Video Generationcited by 237
- 2020DNSMOS: A Non-Intrusive Perceptual Objective Speech Quality metric to evaluate Noise Suppressorscited by 235
- 2022MM-Diffusion: Learning Multi-Modal Diffusion Models for Joint Audio and Video Generationcited by 225
- 2021Efficient Training of Audio Transformers with Patchoutcited by 222
- 2026LTX-2: Efficient Joint Audio-Visual Foundation Modelcited by 213
- 2025Video models are zero-shot learners and reasonerscited by 208
- 2024Stable Audio Opencited by 192
- 2025Open-Sora 2.0: Training a Commercial-Level Video Generation Model in 200kcited by 157
- 2024MMAudio: Taming Multimodal Joint Training for High-Quality Video-to-Audio Synthesiscited by 134
- 2023Diff-Foley: Synchronized Video-to-Audio Synthesis with Latent Diffusion Modelscited by 133
- 2025Meta Audiobox Aesthetics: Unified Automatic Quality Assessment for Speech, Music, and Soundcited by 129
- 2024FoleyCrafter: Bring Silent Videos to Life with Lifelike and Synchronized Soundscited by 109
- 2024Tango 2: Aligning Diffusion-based Text-to-Audio Generations through Direct Preference Optimizationcited by 97
- 2024Synchformer: Efficient Synchronization from Sparse Cuescited by 94
- 2025JavisDiT: Joint Audio-Video Diffusion Transformer with Hierarchical Spatio-Temporal Prior Synchronizationcited by 84
- 2024OpenHumanVid: A Large-Scale High-Quality Dataset for Enhancing Human-Centric Video Generationcited by 77
- 2025Ovi: Twin Backbone Cross-Modal Fusion for Audio-Video Generationcited by 76
- 2024ChronoMagic-Bench: A Benchmark for Metamorphic Evaluation of Text-to-Time-lapse Video Generationcited by 74
- 2024DiTCtrl: Exploring Attention Control in Multi-Modal Diffusion Transformer for Tuning-Free Multi-Prompt Longer Video Generationcited by 73
- 2025LongCat-Video Technical Reportcited by 73
50 most cited of 64 All 64 (JSON)
Built on by
Citing papers per year: 2026: 39.
39 papers.
Sort:
By area:
- 2026Speed by Simplicity: A Single-Stream Architecture for Fast Audio-Video Generative Foundation Modelcited by 23
- 2026Hallo-Live: Real-Time Streaming Joint Audio-Video Avatar Generation with Asynchronous Dual-Stream and Human-Centric Preference Distillationcited by 13
- 2026MSAVBench: Towards Comprehensive and Reliable Evaluation of Multi-Shot Audio-Video Generationcited by 8
- 2026Unison: Harmonizing Motion, Speech, and Sound for Human-Centric Audio-Video Generationcited by 6
- 2026Native Audio-Visual Alignment for Generationcited by 6
- 2026CineDance: Towards Next-Generation Multi-Shot Long-Form Cinematic Audio-Video Generationcited by 5
- 2026Improving Joint Audio-Video Generation with Cross-Modal Context Learningcited by 4
- 2026MMControl: Unified Multi-Modal Control for Joint Audio-Video Generationcited by 4
- 2026InstructAV2AV: Instruction-Guided Audio-Video Joint Editingcited by 4
- 2026MaineCoon: Pursuing A Real-Time Audio-Visual Social World Modelcited by 4
- 2026UnityShots: Memory-Driven Multi-Shot Audio-Video Generation with Boundary-Aware Gatingcited by 4
- 2026Baton: Explicit Semantic Blueprints for Joint Video-Audio Generationcited by 3
27 more
- 2026StreamChar: Long-Horizon Streaming Character Audio-Video Generation with Decoupled Orchestrationcited by 3
- 2026MTAVG-Bench 2.0: Diagnosing Failure Modes of Cinematic Expressiveness in Multi-Talker Audio-Video Generationcited by 3
- 2026VA-Judger: Reward Modeling from Human Preference Feedback for Joint Video-Audio Generationcited by 3
- 2026Talker-T2AV: Joint Talking Audio-Video Generation with Autoregressive Diffusion Modelingcited by 2
- 2026Omni-Customizer: End-to-End MultiModal Customization for Joint Audio-Video Generationcited by 2
- 2026InteracVid: Building a Real Interactive Audio-Visual Response Dataset from Live-Chat Videoscited by 2
- 2026InterDyad: Interactive Dyadic Speech-to-Video Generation by Querying Intermediate Visual Guidancecited by 1
- 2026Efficient Audio-Visual Generation via Synchrony-Aware Cross-Modal Sparse Attentioncited by 1
- 2026Beyond Coherence: Benchmarking Professional Editing-Technique Execution in Multi-Shot Audio-Video Generationcited by 1
- 2026Salt++: Context-Aligned Post-Training for Few-Step Streaming Multimodal Generationcited by 1
- 2026mAVE: A Watermark for Joint Audio-Visual Generation Modelscited by 0
- 2026Tora3: Trajectory-Guided Audio-Video Generation with Physical Coherencecited by 0
- 2026Spatial-Temporal Decoupled Reference Conditioning for Identity-Preserving Text-to-Video Generationcited by 0
- 2026MAVIN: Multi-Shot Audio-Visual Generation with Customized Narrative Controlcited by 0
- 2026Beyond Time Shifts: Adapting Omni-LLM as a Reference-Free Evaluator for Generative Audio-Visual Modelscited by 0
- 2026OmniMate: Open-Ended Real-Time Streaming Audio-Visual Generation for Interactive Avatarscited by 0
- 2026TaoMate: Anchor-Guided Memory Bridging Evolving and Reference States for Real-Time Audio-Video Digital Human Generationcited by 0
- 2026Ripple: Real-Time Streaming Audio-Video Generation With Cross-Modal Recurrent Memorycited by 0
- 2026Vorch-Omni: Multi-Task Orchestration of Sight and Soundcited by 0
- 2026DreamX-Creator: Democratizing Native Audio-Video Generation at 2K Resolutioncited by 0
- 2026TimeSteer: Inference-Time Speech Scheduling in Joint Audio-Visual Diffusion Modelscited by 0
- 2026PRISM-Bench: An Audio-Centric Diagnostic Benchmark for Text-to-Audio-Video Generationcited by 0
- 2026All modalities are equal, but video is more equal: Closing the Cross-Attention Gap in Joint Video Generationcited by 0
- 2026AV-GRPO: Modality-Anchored Decoupling Diffusion Reinforcement Learning for Joint Audio-Video Generationcited by 0
- 2026Joint and Cross-Modal Video-Audio Generation and Editing: A Unified Formulation and Design Taxonomycited by 0
- 2026Hear the World in Stereo: Learning Dynamic Spatial Correspondence for Immersive Joint Video-Audio Generationcited by 0
- 2026From Video Clips to Creation Trajectory: Sora100K for AI-Native Video Creationcited by 0
- 2026From Video Clips to Creation Trajectory: Sora100K for AI-Native Video Creation8 Oct 2026
- 2026Hear the World in Stereo: Learning Dynamic Spatial Correspondence for Immersive Joint Video-Audio Generation30 Sep 2026
- 2026Salt++: Context-Aligned Post-Training for Few-Step Streaming Multimodal Generation29 Sep 2026
- 2026Joint and Cross-Modal Video-Audio Generation and Editing: A Unified Formulation and Design Taxonomy28 Sep 2026
- 2026AV-GRPO: Modality-Anchored Decoupling Diffusion Reinforcement Learning for Joint Audio-Video Generation24 Sep 2026
- 2026All modalities are equal, but video is more equal: Closing the Cross-Attention Gap in Joint Video Generation23 Sep 2026
- 2026Beyond Coherence: Benchmarking Professional Editing-Technique Execution in Multi-Shot Audio-Video Generation8 Sep 2026
- 2026PRISM-Bench: An Audio-Centric Diagnostic Benchmark for Text-to-Audio-Video Generation4 Sep 2026
- 2026TimeSteer: Inference-Time Speech Scheduling in Joint Audio-Visual Diffusion Models1 Sep 2026
- 2026DreamX-Creator: Democratizing Native Audio-Video Generation at 2K Resolution31 Aug 2026
- 2026VA-Judger: Reward Modeling from Human Preference Feedback for Joint Video-Audio Generation19 Aug 2026
- 2026Efficient Audio-Visual Generation via Synchrony-Aware Cross-Modal Sparse Attention16 Aug 2026
27 more
- 2026Vorch-Omni: Multi-Task Orchestration of Sight and Sound6 Aug 2026
- 2026InteracVid: Building a Real Interactive Audio-Visual Response Dataset from Live-Chat Videos2 Aug 2026
- 2026Ripple: Real-Time Streaming Audio-Video Generation With Cross-Modal Recurrent Memory29 Jul 2026
- 2026TaoMate: Anchor-Guided Memory Bridging Evolving and Reference States for Real-Time Audio-Video Digital Human Generation27 Jul 2026
- 2026OmniMate: Open-Ended Real-Time Streaming Audio-Visual Generation for Interactive Avatars25 Jul 2026
- 2026Beyond Time Shifts: Adapting Omni-LLM as a Reference-Free Evaluator for Generative Audio-Visual Models10 Jul 2026
- 2026MAVIN: Multi-Shot Audio-Visual Generation with Customized Narrative Control28 Jun 2026
- 2026UnityShots: Memory-Driven Multi-Shot Audio-Video Generation with Boundary-Aware Gating19 Jun 2026
- 2026MaineCoon: Pursuing A Real-Time Audio-Visual Social World Model16 Jun 2026
- 2026CineDance: Towards Next-Generation Multi-Shot Long-Form Cinematic Audio-Video Generation8 Jun 2026
- 2026Spatial-Temporal Decoupled Reference Conditioning for Identity-Preserving Text-to-Video Generation1 Jun 2026
- 2026Native Audio-Visual Alignment for Generation28 May 2026
- 2026MTAVG-Bench 2.0: Diagnosing Failure Modes of Cinematic Expressiveness in Multi-Talker Audio-Video Generation27 May 2026
- 2026StreamChar: Long-Horizon Streaming Character Audio-Video Generation with Decoupled Orchestration25 May 2026
- 2026Baton: Explicit Semantic Blueprints for Joint Video-Audio Generation24 May 2026
- 2026MSAVBench: Towards Comprehensive and Reliable Evaluation of Multi-Shot Audio-Video Generation19 May 2026
- 2026InstructAV2AV: Instruction-Guided Audio-Video Joint Editing18 May 2026
- 2026Omni-Customizer: End-to-End MultiModal Customization for Joint Audio-Video Generation17 May 2026
- 2026Unison: Harmonizing Motion, Speech, and Sound for Human-Centric Audio-Video Generation9 May 2026
- 2026Hallo-Live: Real-Time Streaming Joint Audio-Video Avatar Generation with Asynchronous Dual-Stream and Human-Centric Preference Distillation26 Apr 2026
- 2026Talker-T2AV: Joint Talking Audio-Video Generation with Autoregressive Diffusion Modeling26 Apr 2026
- 2026MMControl: Unified Multi-Modal Control for Joint Audio-Video Generation21 Apr 2026
- 2026Tora3: Trajectory-Guided Audio-Video Generation with Physical Coherence10 Apr 2026
- 2026InterDyad: Interactive Dyadic Speech-to-Video Generation by Querying Intermediate Visual Guidance24 Mar 2026
- 2026Speed by Simplicity: A Single-Stream Architecture for Fast Audio-Video Generative Foundation Model23 Mar 2026
- 2026Improving Joint Audio-Video Generation with Cross-Modal Context Learning19 Mar 2026
- 2026mAVE: A Watermark for Joint Audio-Visual Generation Models7 Mar 2026
- 2026Hallo-Live: Real-Time Streaming Joint Audio-Video Avatar Generation with Asynchronous Dual-Stream and Human-Centric Preference Distillationcited by 13
- 2026Improving Joint Audio-Video Generation with Cross-Modal Context Learningcited by 4
- 2026MMControl: Unified Multi-Modal Control for Joint Audio-Video Generationcited by 4
- 2026Baton: Explicit Semantic Blueprints for Joint Video-Audio Generationcited by 3
- 2026Talker-T2AV: Joint Talking Audio-Video Generation with Autoregressive Diffusion Modelingcited by 2
- 2026OmniMate: Open-Ended Real-Time Streaming Audio-Visual Generation for Interactive Avatarscited by 0
- 2026Vorch-Omni: Multi-Task Orchestration of Sight and Soundcited by 0
- 2026TimeSteer: Inference-Time Speech Scheduling in Joint Audio-Visual Diffusion Modelscited by 0
- 2026TimeSteer: Inference-Time Speech Scheduling in Joint Audio-Visual Diffusion Models1 Sep 2026
- 2026Vorch-Omni: Multi-Task Orchestration of Sight and Sound6 Aug 2026
- 2026OmniMate: Open-Ended Real-Time Streaming Audio-Visual Generation for Interactive Avatars25 Jul 2026
- 2026Baton: Explicit Semantic Blueprints for Joint Video-Audio Generation24 May 2026
- 2026Hallo-Live: Real-Time Streaming Joint Audio-Video Avatar Generation with Asynchronous Dual-Stream and Human-Centric Preference Distillation26 Apr 2026
- 2026Talker-T2AV: Joint Talking Audio-Video Generation with Autoregressive Diffusion Modeling26 Apr 2026
- 2026MMControl: Unified Multi-Modal Control for Joint Audio-Video Generation21 Apr 2026
- 2026Improving Joint Audio-Video Generation with Cross-Modal Context Learning19 Mar 2026
- 2026Speed by Simplicity: A Single-Stream Architecture for Fast Audio-Video Generative Foundation Modelcited by 23
- 2026MSAVBench: Towards Comprehensive and Reliable Evaluation of Multi-Shot Audio-Video Generationcited by 8
- 2026Unison: Harmonizing Motion, Speech, and Sound for Human-Centric Audio-Video Generationcited by 6
- 2026Native Audio-Visual Alignment for Generationcited by 6
- 2026CineDance: Towards Next-Generation Multi-Shot Long-Form Cinematic Audio-Video Generationcited by 5
- 2026InstructAV2AV: Instruction-Guided Audio-Video Joint Editingcited by 4
- 2026UnityShots: Memory-Driven Multi-Shot Audio-Video Generation with Boundary-Aware Gatingcited by 4
- 2026StreamChar: Long-Horizon Streaming Character Audio-Video Generation with Decoupled Orchestrationcited by 3
- 2026MTAVG-Bench 2.0: Diagnosing Failure Modes of Cinematic Expressiveness in Multi-Talker Audio-Video Generationcited by 3
- 2026Omni-Customizer: End-to-End MultiModal Customization for Joint Audio-Video Generationcited by 2
- 2026InteracVid: Building a Real Interactive Audio-Visual Response Dataset from Live-Chat Videoscited by 2
- 2026InterDyad: Interactive Dyadic Speech-to-Video Generation by Querying Intermediate Visual Guidancecited by 1
12 most cited of 28 All 28 (JSON)
- 2026From Video Clips to Creation Trajectory: Sora100K for AI-Native Video Creation8 Oct 2026
- 2026Hear the World in Stereo: Learning Dynamic Spatial Correspondence for Immersive Joint Video-Audio Generation30 Sep 2026
- 2026Salt++: Context-Aligned Post-Training for Few-Step Streaming Multimodal Generation29 Sep 2026
- 2026Joint and Cross-Modal Video-Audio Generation and Editing: A Unified Formulation and Design Taxonomy28 Sep 2026
- 2026AV-GRPO: Modality-Anchored Decoupling Diffusion Reinforcement Learning for Joint Audio-Video Generation24 Sep 2026
- 2026All modalities are equal, but video is more equal: Closing the Cross-Attention Gap in Joint Video Generation23 Sep 2026
- 2026Beyond Coherence: Benchmarking Professional Editing-Technique Execution in Multi-Shot Audio-Video Generation8 Sep 2026
- 2026PRISM-Bench: An Audio-Centric Diagnostic Benchmark for Text-to-Audio-Video Generation4 Sep 2026
- 2026DreamX-Creator: Democratizing Native Audio-Video Generation at 2K Resolution31 Aug 2026
- 2026Efficient Audio-Visual Generation via Synchrony-Aware Cross-Modal Sparse Attention16 Aug 2026
- 2026InteracVid: Building a Real Interactive Audio-Visual Response Dataset from Live-Chat Videos2 Aug 2026
- 2026Ripple: Real-Time Streaming Audio-Video Generation With Cross-Modal Recurrent Memory29 Jul 2026
12 newest of 28 All 28 (JSON)
Similar, not cited
Close in wording, no citation either way
- 2024LoVA: Long-form Video-to-Audio Generation
- 2025Seedance 1.5 pro: A Native Audio-Visual Joint Generation Foundation Model
- 2026Omni2Sound: Towards Unified Video-Text-to-Audio Generation
- 2026Soundwich: Video Generation with Layered and Controllable Audio
- 2026Echoes Over Time: Unlocking Length Generalization in Video-to-Audio Generation Models
- 2025PhyAVBench: A Challenging Audio Physics-Sensitivity Benchmark for Physically Grounded Text-to-Audio-Video Generation
- 2024Audio-Synchronized Visual Animation
- 2026JavisDiT++: Unified Modeling and Optimization for Joint Audio-Video Generation