DreamX-Creator: Democratizing Native Audio-Video Generation at 2K Resolution
Not on the map. Why?
Abstract (arXiv)
Recent video generators often omit audio or synthesize it in a separate stage, limiting reciprocal modeling of visual dynamics and acoustic events. We present DreamX-Creator 1.0, a compact native joint audio-video generation system centered on a 7B generator. Conditioned on a first frame and a text prompt, the generator jointly denoises modality-specialized audio and video streams. The streams are processed independently in the first half of the network and coupled in the latter half through Gated Cross-Modal Attention, whose token- and head-wise output gates modulate each active cross-modal attention-head output. A unified Audio-Video Data System constructs and filters temporally coherent clips, produces structured multimodal annotations, and organizes clips into capability-oriented data pools. Progressive Joint Training comprises two audio-video pre-training stages followed by High-Quality Finetuning. Audio-Video Reinforcement Learning further post-trains the generator with Modality-Aware Multimodal Feedback that routes video-, audio-, and cross-modal feedback to the corresponding streams. For high-resolution output, our Autoregressive 1-Step 2K Refinement pipeline adapts a bidirectional multi-step teacher into an autoregressive multi-step refiner and distills it into a student requiring one denoising evaluation per temporal chunk. Overall, DreamX-Creator 1.0 achieves native, synchronized audio-video generation with performance competitive with state-of-the-art open-source systems. By releasing our compact 7B generator and 2K Refiner, we seek to democratize native audio-video generation and provide an accessible foundation for future research in unified audio-video generative modeling.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 72 of its 87 references that are in the map. First posted on arXiv 31 Aug 2026.
Builds on
72 of 87 references.
References · 72
- 2022Flow Matching for Generative Modelingcited by 5,702
- 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,822
- 2025DINOv3cited by 1,414
- 2023ImageBind: One Embedding Space To Bind Them Allcited by 1,175
- 2021MUSIQ: Multi-scale Image Quality Transformercited by 1,081
- 2023One-step Diffusion with Distribution Matching Distillationcited by 1,019
- 2023Training Diffusion Models with Reinforcement Learningcited by 982
- 2023Diffusion Model Alignment Using Direct Preference Optimizationcited by 790
- 2024Improved Distribution Matching Distillation for Fast Image Synthesiscited by 684
- 2025Flow-GRPO: Training Flow Matching Models via Online RLcited by 661
- 2020VGGSound: A Large-scale Audio-Visual Datasetcited by 648
- 2025Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusioncited by 625
38 more
- 2022MANIQA: Multi-dimension Attention Network for No-Reference Image Quality Assessmentcited by 522
- 2023Q-Align: Teaching LMMs for Visual Scoring via Discrete Text-Defined Levelscited by 503
- 2025Qwen3-Omni Technical Reportcited by 497
- 2021BasicVSR++: Improving Video Super-Resolution with Enhanced Propagation and Alignmentcited by 342
- 2025Gated Attention for Large Language Models: Non-linearity, Sparsity, and Attention-Sink-Freecited by 283
- 2024Fast High-Resolution Image Synthesis with Latent Adversarial Diffusion Distillationcited by 279
- 2022Unifying Flow, Stereo and Depth Estimationcited by 275
- 2023Using Human Feedback to Fine-tune Diffusion Models without Any Reward Modelcited by 252
- 2026LTX-2: Efficient Joint Audio-Visual Foundation Modelcited by 236
- 2025DiffusionNFT: Online Diffusion Reinforcement with Forward Processcited by 230
- 2022MM-Diffusion: Learning Multi-Modal Diffusion Models for Joint Audio and Video Generationcited by 227
- 2022VRT: A Video Restoration Transformercited by 220
- 2026Causal Forcing: Autoregressive Diffusion Distillation Done Right for High-Quality Real-Time Interactive Video Generationcited by 184
- 2025Meta Audiobox Aesthetics: Unified Automatic Quality Assessment for Speech, Music, and Soundcited by 137
- 2024Koala-36M: A Large-scale Video Dataset Improving Consistency between Fine-grained Conditions and Video Contentcited by 135
- 2021Investigating Tradeoffs in Real-World Video Super-Resolutioncited by 134
- 2023Diff-Foley: Synchronized Video-to-Audio Synthesis with Latent Diffusion Modelscited by 134
- 2024MMAudio: Taming Multimodal Joint Training for High-Quality Video-to-Audio Synthesiscited by 134
- 2023Upscale-A-Video: Temporal-Consistent Diffusion Model for Real-World Video Super-Resolutioncited by 132
- 2025OmniHuman-1: Rethinking the Scaling-Up of One-Stage Conditioned Human Animation Modelscited by 129
- 2026Qwen3-ASR Technical Reportcited by 118
- 2024Seeing and Hearing: Open-domain Visual-Audio Generation with Diffusion Latent Alignerscited by 111
- 2024FoleyCrafter: Bring Silent Videos to Life with Lifelike and Synchronized Soundscited by 110
- 2024VideoDPO: Omni-Preference Alignment for Video Diffusion Generationcited by 108
- 2024Video Diffusion Alignment via Reward Gradientscited by 107
- 2024Synchformer: Efficient Synchronization from Sparse Cuescited by 97
- 2025JavisDiT: Joint Audio-Video Diffusion Transformer with Hierarchical Spatio-Temporal Prior Synchronizationcited by 88
- 2025Ovi: Twin Backbone Cross-Modal Fusion for Audio-Video Generationcited by 83
- 2024OpenHumanVid: A Large-Scale High-Quality Dataset for Enhancing Human-Centric Video Generationcited by 77
- 2024VEnhancer: Generative Space-Time Enhancement for Video Generationcited by 64
- 2025SeedVR: Seeding Infinity in Diffusion Transformer Towards Generic Video Restorationcited by 62
- 2025Wan-S2V: Audio-Driven Cinematic Video Generationcited by 61
- 2025UniVerse-1: Unified Audio-Video Generation via Stitching of Expertscited by 61
- 2025VMBench: A Benchmark for Perception-Aligned Video Motion Generationcited by 49
- 2025FlashVSR: Towards Real-Time Diffusion-Based Streaming Video Super-Resolutioncited by 43
- 2026MOVA: Towards Scalable and Synchronized Video-Audio Generationcited by 43
- 2024AV-DiT: Efficient Audio-Visual Diffusion Transformer for Joint Audio and Video Generationcited by 41
- 2026DreamX-World 1.0: A General-Purpose Interactive World Modelcited by 40
50 most cited of 72 All 72 (JSON)
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Seedance 1.5 pro: A Native Audio-Visual Joint Generation Foundation Model
- 2026Efficient Audio-Visual Generation via Synchrony-Aware Cross-Modal Sparse Attention
- 2026Diffusion Models for Joint Audio-Video Generation
- 2025Does Hearing Help Seeing? Investigating Audio-Video Joint Denoising for Video Generation
- 2026Soundwich: Video Generation with Layered and Controllable Audio
- 2025DreamFoley: Scalable VLMs for High-Fidelity Video-to-Audio Generation
- 2026HarmoniDPO: Video-guided Audio Generation via Preference-Optimized Diffusion
- 2026AVTok: 1D Unified Tokenization for Holistic Audio-Video Generation