Mamoda2.5: Enhancing Unified Multimodal Model with DiT-MoE
On the map: Mixture-of-experts → Multimodal MoE models
Abstract (arXiv)
We present Mamoda2.5, a unified AR-Diffusion framework that seamlessly integrates multimodal understanding and generation within a single architecture. To efficiently enhance the model's generation capability, we equip the Diffusion Transformer backbone with a fine-grained Mixture-of-Experts (MoE) design (128 experts, Top-8 routing), yielding a 25B-parameter model that activates only 3B parameters, significantly reducing training costs while scaling up the model capacity. Mamoda2.5 achieves top-tier generation performance on VBench 2.0 and sets a new record in video editing quality, surpassing evaluated open-source models and matching the performance of current top-tier proprietary models, including the Kling O1 on OpenVE-Bench. Furthermore, we introduce a joint few-step distillation and reinforcement learning framework that compresses the 30-step editing model into a 4-step model and greatly accelerates model inference. Compared to open-source baselines, Mamoda2.5 achieves up to 95.9× faster video editing inference. In real-world applications, Mamoda2.5 has been successfully deployed for content moderation and creative restoration tasks in advertising scenarios, achieving a 98% success rate in internal advertising video editing scenario.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Multimodal MoE models, one of 12 directions in Mixture-of-experts. Builds on 61 of its 76 references that are in the map. First posted on arXiv 4 May 2026.
Builds on
61 of 76 references.
References · 61
- 2022Scalable Diffusion Models with Transformerscited by 5,931
- 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
- 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
- 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
- 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
- 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,014
- 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,536
- 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,755
- 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,597
- 2023One-step Diffusion with Distribution Matching Distillationcited by 978
- 2023GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignmentcited by 946
- 2025Emerging Properties in Unified Multimodal Pretrainingcited by 855
38 more
- 2025Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scalingcited by 787
- 2024Show-o: One Single Transformer to Unify Multimodal Understanding and Generationcited by 734
- 2021ByT5: Towards a token-free future with pre-trained byte-to-byte modelscited by 504
- 2025VACE: All-in-One Video Creation and Editingcited by 467
- 2024ELLA: Equip Diffusion Models with LLM for Enhanced Semantic Alignmentcited by 455
- 2023TokenFlow: Consistent Diffusion Features for Consistent Video Editingcited by 440
- 2025Step1X-Edit: A Practical Framework for General Image Editingcited by 417
- 2022DeepSpeed-MoE: Advancing Mixture-of-Experts Inference and Training to Power Next-Generation AI Scalecited by 400
- 2024OmniGen: Unified Image Generationcited by 388
- 2025BLIP3-o: A Family of Fully Open Unified Multimodal Models-Architecture, Training and Datasetcited by 363
- 2024SANA: Efficient High-Resolution Image Synthesis with Linear Diffusion Transformerscited by 331
- 2025VBench-2.0: Advancing Video Generation Benchmark Suite for Intrinsic Faithfulnesscited by 289
- 2025Seedance 1.0: Exploring the Boundaries of Video Generation Modelscited by 247
- 2025Seedream 4.0: Toward Next-generation Multimodal Image Generationcited by 233
- 2025ImgEdit: A Unified Image Editing Dataset and Benchmarkcited by 225
- 2022Sparse Upcycling: Training Mixture-of-Experts from Dense Checkpointscited by 207
- 2025DiffusionNFT: Online Diffusion Reinforcement with Forward Processcited by 204
- 2024Auxiliary-Loss-Free Load Balancing Strategy for Mixture-of-Expertscited by 191
- 2024Vidu: a Highly Consistent, Dynamic and Skilled Text-to-Video Generator with Diffusion Modelscited by 159
- 2025Seedream 3.0 Technical Reportcited by 146
- 2025HiDream-I1: A High-Efficient Image Generative Foundation Model with Sparse Diffusion Transformercited by 131
- 2023Space-Time Diffusion Features for Zero-Shot Text-Driven Motion Transfercited by 124
- 2023VidToMe: Video Token Merging for Zero-Shot Video Editingcited by 111
- 2025Mogao: An Omni Foundation Model for Interleaved Multi-Modal Generationcited by 108
- 2025UniVideo: Unified Understanding, Generation, and Editing for Videoscited by 85
- 2025Scaling Instruction-Based Video Editing with a High-Quality Synthetic Datasetcited by 76
- 2024Scaling Diffusion Transformers to 16 Billion Parameterscited by 65
- 2025UniWorld-V1: High-Resolution Semantic Encoders for Unified Visual Understanding and Generationcited by 64
- 2025InsViE-1M: Effective Instruction-based Video Editing with Elaborate Dataset Constructioncited by 62
- 2024USP: A Unified Sequence Parallelism Approach for Long Context Generative AIcited by 58
- 2025VideoGrain: Modulating Space-Time Attention for Multi-grained Video Editingcited by 50
- 2025OpenVE-3M: A Large-Scale High-Quality Dataset for Instruction-Guided Video Editingcited by 50
- 2025Distribution Matching Distillation Meets Reinforcement Learningcited by 43
- 2025Omni-Video: Democratizing Unified Video Understanding and Generationcited by 39
- 2025In-Context Learning with Unpaired Clips for Instruction-based Video Editingcited by 34
- 2026Kiwi-Edit: Versatile Video Editing via Instruction and Reference Guidancecited by 34
- 2025Bee: A High-Quality Corpus and Full-Stack Suite to Unlock Advanced Fully Open MLLMscited by 27
- 2026VINO: A Unified Visual Generator with Interleaved OmniModal Contextcited by 26
50 most cited of 61 All 61 (JSON)
Built on by
Citing papers per year: 2026: 3.
3 papers.
- 2026Toward Native Multimodal Modeling: A Roadmapcited by 6
- 2026Sparse Mixture-of-Experts Routing in Visual Diffusion Transformers:Diagnosis, Boundary Calibration and Evolutionary Roadmap from Routing Collapse to Selective Deadlockcited by 2
- 2026MoECa: Aligning Feature Reuse with Expert Decomposition in Diffusion Transformerscited by 0
- 2026MoECa: Aligning Feature Reuse with Expert Decomposition in Diffusion Transformers14 Jun 2026
- 2026Toward Native Multimodal Modeling: A Roadmap25 May 2026
- 2026Sparse Mixture-of-Experts Routing in Visual Diffusion Transformers:Diagnosis, Boundary Calibration and Evolutionary Roadmap from Routing Collapse to Selective Deadlock12 May 2026
Similar, not cited
Close in wording, no citation either way
- 2026LoomVideo: Unifying Multimodal Inputs into Video Generation and Editing
- 2025MMaDA: Multimodal Large Diffusion Language Models
- 2025Emu3.5: Native Multimodal Models are World Learners
- 2025Hyper-Bagel: A Unified Acceleration Framework for Multimodal Understanding and Generation
- 2025Lavida-O: Elastic Large Masked Diffusion Models for Unified Multimodal Understanding and Generation
- 2025InstructX: Towards Unified Visual Editing with MLLM Guidance
- 2026LongLive-Plug: Once-for-All Distillation for Video Generation
- 2026Tele-Omni: a Unified Multimodal Framework for Video Generation and Editing