mlsanityData to 7 Oct 2026

Mogao: An Omni Foundation Model for Interleaved Multi-Modal Generation

arXiv 2505.05472 ↗First posted 8 May 2025cs.CVcited by 108

Not on the map. Why?

Cited by 24 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Recent progress in unified models for image understanding and generation has been impressive, yet most approaches remain limited to single-modal generation conditioned on multiple modalities. In this paper, we present Mogao, a unified framework that advances this paradigm by enabling interleaved multi-modal generation through a causal approach. Mogao integrates a set of key technical improvements in architecture design, including a deep-fusion design, dual vision encoders, interleaved rotary position embeddings, and multi-modal classifier-free guidance, which allow it to harness the strengths of both autoregressive models for text generation and diffusion models for high-quality image synthesis. These practical improvements also make Mogao particularly effective to process interleaved sequences of text and images arbitrarily. To further unlock the potential of unified models, we introduce an efficient training strategy on a large-scale, in-house dataset specifically curated for joint text and image generation. Extensive experiments show that Mogao not only achieves state-of-the-art performance in multi-modal understanding and text-to-image generation, but also excels in producing high-quality, coherent interleaved outputs. Its emergent capabilities in zero-shot image editing and compositional generation highlight Mogao as a practical omni-modal foundation model, paving the way for future development and scaling the unified multi-modal systems.

Key facts. Cited by 108 papers in the map (data through 7 Oct 2026). Builds on 78 of its 92 references that are in the map. First posted on arXiv 8 May 2025.

Builds on

78 of 92 references.

References · 78

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  5. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  6. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  7. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  8. 2023Visual Instruction Tuningcited by 7,792
  9. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  10. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  11. 2022Scalable Diffusion Models with Transformerscited by 5,931
  12. 2022Classifier-Free Diffusion Guidancecited by 5,717
38 more
  1. 2025Qwen2.5-VL Technical Reportcited by 5,418
  2. 2024Qwen2.5 Technical Reportcited by 4,713
  3. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  4. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  5. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  6. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
  7. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
  8. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
  9. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,550
  10. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  11. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,048
  12. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
  13. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,591
  14. 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,338
  15. 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,105
  16. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,044
  17. 2023Evaluating Object Hallucination in Large Vision-Language Modelscited by 1,696
  18. 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,624
  19. 2019MMDetection: Open MMLab Detection Toolbox and Benchmarkcited by 1,379
  20. 2022MaskGIT: Masked Generative Image Transformercited by 1,082
  21. 2024Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Predictioncited by 1,005
  22. 2023GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignmentcited by 946
  23. 2023SEED-Bench: Benchmarking Multimodal LLMs with Generative Comprehensioncited by 929
  24. 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 904
  25. 2025Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scalingcited by 787
  26. 2024Emu3: Next-Token Prediction is All You Needcited by 761
  27. 2024Autoregressive Model Beats Diffusion: Llama for Scalable Image Generationcited by 760
  28. 2024Show-o: One Single Transformer to Unify Multimodal Understanding and Generationcited by 734
  29. 2024DeepSeek-VL: Towards Real-World Vision-Language Understandingcited by 718
  30. 2024Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Thinkcited by 666
  31. 2024Autoregressive Image Generation without Vector Quantizationcited by 644
  32. 2024Transfusion: Predict the Next Token and Diffuse Images with One Multi-Modal Modelcited by 468
  33. 2024Evaluating Text-to-Visual Generation with Image-to-Text Generationcited by 465
  34. 2024Janus: Decoupling Visual Encoding for Unified Multimodal Understanding and Generationcited by 447
  35. 2023Generative Multimodal Models are In-Context Learnerscited by 446
  36. 2024OmniGen: Unified Image Generationcited by 388
  37. 2022Pix2Struct: Screenshot Parsing as Pretraining for Visual Language Understandingcited by 360
  38. 2024SEED-X: Multimodal Models with Unified Multi-granularity Comprehension and Generationcited by 328

50 most cited of 78 All 78 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 52; 2026: 56.

108 papers.

Sort:

By area:

  1. 2025Show-o2: Improved Native Unified Multimodal Modelscited by 242
  2. 2025OmniGen2: Towards Instruction-Aligned Multimodal Generationcited by 241
  3. 2025Thinking with Images for Multimodal Reasoning: Foundations, Methods, and Future Frontierscited by 193
  4. 2026Seedance 2.0: Advancing Video Generation for World Complexitycited by 142
  5. 2025HunyuanImage 3.0 Technical Reportcited by 138
  6. 2025X-Omni: Reinforcement Learning Makes Discrete Autoregressive Image Generative Models Great Againcited by 105
  7. 2025F1: A Vision-Language-Action Model Bridging Understanding and Generation to Actionscited by 85
  8. 2025UniVideo: Unified Understanding, Generation, and Editing for Videoscited by 85
  9. 2025Lumina-DiMOO: An Omni Diffusion Large Language Model for Multi-Modal Generation and Understandingcited by 83
  10. 2025PixNerd: Pixel Neural Field Diffusioncited by 75
  11. 2025RewardDance: Reward Scaling in Visual Generationcited by 63
  12. 2025NextStep-1: Toward Autoregressive Image Generation with Continuous Tokens at Scalecited by 55
38 more
  1. 2025DeCo: Frequency-Decoupled Pixel Diffusion for End-to-End Image Generationcited by 55
  2. 2025SeedEdit 3.0: Fast and High-Quality Generative Image Editingcited by 52
  3. 2025OneCAT: Decoder-Only Auto-Regressive Model for Unified Understanding and Generationcited by 50
  4. 2025UniLiP: Adapting CLIP for Unified Multimodal Understanding, Generation and Editingcited by 43
  5. 2025Interleaving Reasoning for Better Text-to-Image Generationcited by 40
  6. 2025Omni-Video: Democratizing Unified Video Understanding and Generationcited by 39
  7. 2025TUNA: Taming Unified Visual Representations for Native Unified Multimodal Modelscited by 34
  8. 2025EO-1: An Open Unified Embodied Foundation Model for General Robot Controlcited by 30
  9. 2026LongCat-Next: Lexicalizing Modalities as Discrete Tokenscited by 30
  10. 2026SenseNova-U1: Unifying Multimodal Understanding and Generation with NEO-unify Architecturecited by 27
  11. 2026VINO: A Unified Visual Generator with Interleaved OmniModal Contextcited by 26
  12. 2025ReasonEdit: Towards Reasoning-Enhanced Image Editing Modelscited by 24
  13. 2025Lumos-1: On Autoregressive Video Generation with Discrete Diffusion from a Unified Model Perspectivecited by 22
  14. 2025Uni-X: Mitigating Modality Conflict with a Two-End-Separated Architecture for Unified Multimodal Modelscited by 18
  15. 2025Generative Universal Verifier as Multimodal Meta-Reasonercited by 17
  16. 2025FARMER: Flow AutoRegressive Transformer over Pixelscited by 17
  17. 2026Tuna-2: Pixel Embeddings Beat Vision Encoders for Multimodal Understanding and Generationcited by 17
  18. 2025From Pixels to Words -- Towards Native Vision-Language Primitives at Scalecited by 15
  19. 2025Hyper-Bagel: A Unified Acceleration Framework for Multimodal Understanding and Generationcited by 13
  20. 2025Revisiting Multimodal Positional Encoding in Vision-Language Modelscited by 13
  21. 2026NextFlow: Unified Sequential Modeling Activates Multimodal Understanding and Generationcited by 13
  22. 2025UniMedVL: Unifying Medical Multimodal Understanding and Generation through Observation-Knowledge-Analysiscited by 12
  23. 2026Demystifying Video Reasoningcited by 12
  24. 2026LLaDA2.0-Uni: Unifying Multimodal Understanding and Generation with Diffusion Large Language Modelcited by 12
  25. 2025UniFork: Exploring Modality Alignment for Unified Multimodal Understanding and Generationcited by 11
  26. 2025Uniform Discrete Diffusion with Metric Path for Video Generationcited by 11
  27. 2025HBridge: H-Shape Bridging of Heterogeneous Experts for Unified Multimodal Understanding and Generationcited by 11
  28. 2026UniReason 1.0: A Unified Reasoning Framework for World Knowledge Aligned Image Generation and Editingcited by 11
  29. 2026UniGRPO: Unified Policy Optimization for Reasoning-Driven Visual Generationcited by 11
  30. 2026Lance: Unified Multimodal Modeling by Multi-Task Synergycited by 11
  31. 2025TBAC-UniImage: Unified Understanding and Generation by Ladder-Side Diffusion Tuningcited by 10
  32. 2026PixelGen: Improving Pixel Diffusion with Perceptual Supervisioncited by 10
  33. 2026LLaMo: Scaling Pretrained Language Models for Unified Motion Understanding and Generation with Continuous Autoregressive Tokenscited by 10
  34. 2025Does Understanding Inform Generation in Unified Multimodal Models? From Analysis to Path Forwardcited by 9
  35. 2025Video-GPT via Next Clip Diffusioncited by 8
  36. 2025Locality-aware Parallel Decoding for Efficient Autoregressive Image Generationcited by 8
  37. 2025OCRGenBench: A Comprehensive Benchmark for Evaluating OCR Generative Capabilitiescited by 8
  38. 2025Bifrost-1: Bridging Multimodal LLMs and Diffusion Models with Patch-level CLIP Latentscited by 8

50 most cited of 108 All 108 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Exploring the Deep Fusion of Large Language Models and Diffusion Transformers for Text-to-Image Synthesis
  2. 2026UniGen-AR: Unifying Visual Generation with Auto-Regressive Modeling
  3. 2025Muddit: Liberating Generation Beyond Text-to-Image with a Unified Discrete Diffusion Model
  4. 2025MMGen: Unified Multi-modal Image Generation and Understanding in One Go
  5. 2026Unified Text-Image Generation with Weakness-Targeted Post-Training
  6. 2023MultiFusion: Fusing Pre-Trained Models for Multi-Lingual, Multi-Modal Image Generation
  7. 2022Write and Paint: Generative Vision-Language Models are Unified Modal Learners
  8. 2025Lavida-O: Elastic Large Masked Diffusion Models for Unified Multimodal Understanding and Generation