mlsanityData to 14 Oct 2026

Muddit: Liberating Generation Beyond Text-to-Image with a Unified Discrete Diffusion Model

arXiv 2505.23606 ↗First posted 29 May 2025cs.LG cs.CVcited by 55

On the map: Diffusion models → Image generation, text-to-image and editing

Abstract (arXiv)

Unified generation models aim to handle diverse tasks across modalities -- such as text generation, image generation, and vision-language reasoning -- within a single architecture and decoding paradigm. Autoregressive unified models suffer from slow inference due to sequential decoding, and non-autoregressive unified models suffer from weak generalization due to limited pretrained backbones. We introduce the second-generation Meissonic: Muddit, a unified discrete diffusion transformer that enables fast and parallel generation across both text and image modalities. Unlike prior unified diffusion models trained from scratch, Muddit integrates strong visual priors from a pretrained text-to-image backbone with a lightweight text decoder, enabling flexible and high-quality multimodal generation under a unified architecture. Empirical results show that Muddit achieves competitive or superior performance compared to significantly larger autoregressive models in both quality and efficiency. The work highlights the potential of purely discrete diffusion, when equipped with strong visual priors, as a scalable and effective backbone for unified generation.

Key facts. Cited by 55 papers in the map (data through 14 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 57 of its 76 references that are in the map. First posted on arXiv 29 May 2025.

Builds on

57 of 76 references.

Closest ancestors · 12

Same direction, newest first

  1. 2025MMaDA: Multimodal Large Diffusion Language Models21 May
  2. 2025BLIP3-o: A Family of Fully Open Unified Multimodal Models-Architecture, Training and Dataset14 May
  3. 2025An Empirical Study of GPT-4o Image Generation Capabilities8 Apr
  4. 2025Unified Multimodal Discrete Diffusion26 Mar
  5. 2025Multimodal Representation Alignment for Image Generation: Text-Image Interleaved Control Is Easier Than You Think27 Feb
  6. 2024Dual Diffusion for Unified Image Generation and Understanding31 Dec
  7. 2024JanusFlow: Harmonizing Autoregression and Rectified Flow for Unified Multimodal Understanding and Generation12 Nov
  8. 2024Meissonic: Revitalizing Masked Generative Transformers for Efficient High-Resolution Text-to-Image Synthesis10 Oct
  9. 2024Emu3: Next-Token Prediction is All You Need27 Sep
  10. 2024VILA-U: a Unified Foundation Model Integrating Visual Understanding and Generation6 Sep
  11. 2023GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignment17 Oct
  12. 2022Hierarchical Text-Conditional Image Generation with CLIP Latents13 Apr

Other references · 45

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  5. 2023Segment Anythingcited by 8,234
  6. 2023Visual Instruction Tuningcited by 7,869
  7. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  8. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  9. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
  10. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
  11. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
  12. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,152
33 more
  1. 2015VQA: Visual Question Answeringcited by 2,506
  2. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,375
  3. 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,141
  4. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,080
  5. 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answeringcited by 1,919
  6. 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,652
  7. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,599
  8. 2021Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Conceptscited by 1,216
  9. 2022MaskGIT: Masked Generative Image Transformercited by 1,099
  10. 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 927
  11. 2025Large Language Diffusion Modelscited by 912
  12. 2023PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesiscited by 864
  13. 2024Simple and Effective Masked Diffusion Language Modelscited by 848
  14. 2025Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scalingcited by 812
  15. 2024Autoregressive Model Beats Diffusion: Llama for Scalable Image Generationcited by 777
  16. 2024Show-o: One Single Transformer to Unify Multimodal Understanding and Generationcited by 752
  17. 2023Discrete Diffusion Modeling by Estimating the Ratios of the Data Distributioncited by 745
  18. 2024Autoregressive Image Generation without Vector Quantizationcited by 651
  19. 2024Simplified and Generalized Masked Diffusion for Discrete Datacited by 626
  20. 2024Transfusion: Predict the Next Token and Diffuse Images with One Multi-Modal Modelcited by 483
  21. 2024Your Absorbing Discrete Diffusion Secretly Models the Conditional Distributions of Clean Datacited by 381
  22. 2024SEED-X: Multimodal Models with Unified Multi-granularity Comprehension and Generationcited by 331
  23. 2023DreamLLM: Synergistic Multimodal Comprehension and Creationcited by 297
  24. 2025Transfer between Modalities with MetaQueriescited by 236
  25. 2024MetaMorph: Multimodal Understanding and Generation via Instruction Tuningcited by 206
  26. 2022Chart-to-Text: A Large-Scale Benchmark for Chart Summarizationcited by 169
  27. 2023Emu: Generative Pretraining in Multimodalitycited by 157
  28. 2024Lumina-mGPT: Illuminate Flexible Photorealistic Text-to-Image Generation with Multimodal Generative Pretrainingcited by 150
  29. 2024Fluid: Scaling Autoregressive Text-to-image Generative Models with Continuous Tokenscited by 147
  30. 2025dKV-Cache: The Cache for Diffusion Language Modelscited by 143
  31. 2024ILLUME: Illuminating Your LLMs to See, Draw, and Self-Enhancecited by 64
  32. 2024MonoFormer: One Transformer for Both Diffusion and Autoregressioncited by 46
  33. 2024Vision Superalignment: Weak-to-Strong Generalization for Vision Foundation Modelscited by 28

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 23; 2026: 32.

55 papers.

Sort:

By area:

  1. 2025Show-o2: Improved Native Unified Multimodal Modelscited by 250
  2. 2025A Survey on Diffusion Language Modelscited by 78
  3. 2025Discrete Diffusion in Large Language and Multimodal Models: A Surveycited by 61
  4. 2025Lumos-1: On Autoregressive Video Generation with Discrete Diffusion from a Unified Model Perspectivecited by 22
  5. 2025MDNS: Masked Diffusion Neural Sampler via Stochastic Optimal Controlcited by 21
  6. 2025Enhancing Reasoning for Diffusion LLMs via Distribution Matching Policy Optimizationcited by 20
  7. 2026Tuna-2: Pixel Embeddings Beat Vision Encoders for Multimodal Understanding and Generationcited by 18
  8. 2025OneFlow: Concurrent Mixed-Modal and Interleaved Generation with Edit Flowscited by 17
  9. 2025Lavida-O: Elastic Large Masked Diffusion Models for Unified Multimodal Understanding and Generationcited by 15
  10. 2025TR2-D2: Tree Search Guided Trajectory-Aware Fine-Tuning for Discrete Diffusioncited by 14
  11. 2025SDAR-VL: Stable and Efficient Block-wise Diffusion for Vision-Language Understandingcited by 13
  12. 2025MM-ACT: Learn from Multimodal Parallel Generation to Actcited by 12
38 more
  1. 2025Does Understanding Inform Generation in Unified Multimodal Models? From Analysis to Path Forwardcited by 10
  2. 2026UEval: A Benchmark for Unified Multimodal Generationcited by 9
  3. 2026Prism: Efficient Test-Time Scaling via Hierarchical Search and Self-Verification for Discrete Diffusion Language Modelscited by 7
  4. 2026LaViDa-R1: Advancing Reasoning for Unified Multimodal Diffusion Language Modelscited by 7
  5. 2026Evolution of Video Generative Foundationscited by 7
  6. 2025Soft-Di[M]O: Improving One-Step Discrete Image Generation with Soft Embeddingscited by 6
  7. 2025Sparse-LaViDa: Sparse Multimodal Discrete Diffusion Language Modelscited by 6
  8. 2025From Masks to Worlds: A Hitchhiker's Guide to World Modelscited by 5
  9. 2026Dynin-Omni: Omnimodal Unified Large Diffusion Language Modelcited by 5
  10. 2025Emotional Support with LLM-based Empathetic Dialogue Generationcited by 4
  11. 2025RecTok: Reconstruction Distillation along Rectified Flowcited by 4
  12. 2025UniModel: A Visual-Only Framework for Unified Multimodal Understanding and Generationcited by 3
  13. 2025Masked Diffusion Models are Secretly Learned-Order Autoregressive Modelscited by 3
  14. 2026FlashAR: Efficient Post-Training Acceleration for Autoregressive Image Generationcited by 3
  15. 2026Transferability Between Understanding and Generation in Unified Multimodal Modelscited by 3
  16. 2025Does Hearing Help Seeing? Investigating Audio-Video Joint Denoising for Video Generationcited by 2
  17. 2025dMLLM-TTS: Self-Verified and Efficient Test-Time Scaling for Diffusion Multi-Modal Large Language Modelscited by 2
  18. 2026IMTBench: A Multi-Scenario Cross-Modal Collaborative Evaluation Benchmark for In-Image Machine Translationcited by 2
  19. 2026Know3D: Prompting 3D Generation with Knowledge from Vision-Language Modelscited by 2
  20. 2025ViewMask-1-to-3: Multi-View Consistent Image Generation via Multimodal Discrete Diffusion Modelscited by 1
  21. 2025FlowBind: Efficient Any-to-Any Generation with Bidirectional Flowscited by 1
  22. 2026AEGIS: Exploring the Limit of World Knowledge Capabilities for Unified Mulitmodal Modelscited by 1
  23. 2026Best of Both Worlds: Multimodal Reasoning and Generation via Unified Discrete Flow Matchingcited by 1
  24. 2026GIDE: Unlocking Diffusion LLMs for Precise Training-Free Image Editingcited by 1
  25. 2026Sketch Then Paint: Hierarchical Reinforcement Learning for Diffusion Multi-Modal Large Language Modelscited by 1
  26. 2026UniTranslator: A Unified Multi-modal Framework for End-to-end In-Image Machine Translationcited by 1
  27. 2026Illuminating Unified Multimodal Model for Free-form Interleaved Text-Image Generationcited by 1
  28. 2026PixelUMM: Encoder-Free Unified Image and Video Understanding and Generationcited by 1
  29. 2026Towards Customized Multimodal Role-Playcited by 0
  30. 2026SynerMedGen: Synergizing Medical Multimodal Understanding with Generation via Task Alignmentcited by 0
  31. 2026FullFlow: Upgrading Text-to-Image Flow Matching Models for Bidirectional Vision--Language Generationcited by 0
  32. 2026Mind-Omni: A Unified Multi-Task Framework for Brain-Vision-Language Modeling via Discrete Diffusioncited by 0
  33. 2026UniCanvas: A Diffusion-base Unified Model for Text-in-Image Joint Generationcited by 0
  34. 2026IMUG-Bench: Benchmarking Unified Multimodal Models on Interleaved Understanding and Generationcited by 0
  35. 2026Efficient Reinforcement for Visual-Textual Thinking with Discrete Diffusion Modelcited by 0
  36. 2026Nemotron-Labs-Diffusion-Image: Advancing Masked Discrete Diffusion for High-Resolution Image Synthesiscited by 0
  37. 2026Argus-Unified: Towards A Compact and Economical Unified Model for Image Understanding and Generationcited by 0
  38. 2026Visual Information-Guided Parallel Decoding for Diffusion Multimodal Large Language Modelscited by 0

50 most cited of 55 All 55 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026High-Fidelity Text-to-Image Generation from Pre-Trained Vision-Language Models via Distribution-Conditioned Diffusion Decoding
  2. 2025Decoder-Only LLMs are Better Controllers for Diffusion Models
  3. 2024ACDiT: Interpolating Autoregressive Conditional Modeling and Diffusion Transformer
  4. 2025Visual Autoregressive Models Beat Diffusion Models on Inference Time Scaling
  5. 2026TextLDM: Language Modeling with Continuous Latent Diffusion
  6. 2026NextFlow: Unified Sequential Modeling Activates Multimodal Understanding and Generation
  7. 2025DiffusionVL: Translating Any Autoregressive Models into Diffusion Vision Language Models
  8. 2026LLaDA2.0-Uni: Unifying Multimodal Understanding and Generation with Diffusion Large Language Model