# Muddit: Liberating Generation Beyond Text-to-Image with a Unified Discrete Diffusion Model

arXiv: https://arxiv.org/abs/2505.23606 · HTML: https://mlsanity.com/paper/2505.23606/

## Key facts

- Cited by 55 papers in the map (data through 14 Oct 2026).
- Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models.
- Builds on 57 of its 76 references that are in the map.
- First posted on arXiv 29 May 2025.

## Placed in

- Diffusion models / [Image generation, text-to-image and editing](https://mlsanity.com/diffusion/image-generation-text-to-image-and-editing/)

## Builds on (most cited first, 30 of 57)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models](https://mlsanity.com/paper/2301.12597/) (2023)
- [Flamingo: a Visual Language Model for Few-Shot Learning](https://mlsanity.com/paper/2204.14198/) (2022)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [Scaling Rectified Flow Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2403.03206/) (2024)
- [LAION-5B: An open large-scale dataset for training next generation image-text models](https://mlsanity.com/paper/2210.08402/) (2022)
- [VQA: Visual Question Answering](https://mlsanity.com/paper/1505.00468/) (2015)
- [BEiT: BERT Pre-Training of Image Transformers](https://mlsanity.com/paper/2106.08254/) (2021)
- [MMBench: Is Your Multi-modal Model an All-around Player?](https://mlsanity.com/paper/2307.06281/) (2023)
- [MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI](https://mlsanity.com/paper/2311.16502/) (2023)
- [Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering](https://mlsanity.com/paper/2209.09513/) (2022)
- [MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models](https://mlsanity.com/paper/2306.13394/) (2023)
- [Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling](https://mlsanity.com/paper/2412.05271/) (2024)
- [Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Concepts](https://mlsanity.com/paper/2102.08981/) (2021)
- [MaskGIT: Masked Generative Image Transformer](https://mlsanity.com/paper/2202.04200/) (2022)
- [GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignment](https://mlsanity.com/paper/2310.11513/) (2023)
- [Chameleon: Mixed-Modal Early-Fusion Foundation Models](https://mlsanity.com/paper/2405.09818/) (2024)
- [Large Language Diffusion Models](https://mlsanity.com/paper/2502.09992/) (2025)
- [PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesis](https://mlsanity.com/paper/2310.00426/) (2023)
- [Simple and Effective Masked Diffusion Language Models](https://mlsanity.com/paper/2406.07524/) (2024)
- [Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scaling](https://mlsanity.com/paper/2501.17811/) (2025)
- [Emu3: Next-Token Prediction is All You Need](https://mlsanity.com/paper/2409.18869/) (2024)
- [Autoregressive Model Beats Diffusion: Llama for Scalable Image Generation](https://mlsanity.com/paper/2406.06525/) (2024)

## Built on by (most cited first, 30 of 55)

- [Show-o2: Improved Native Unified Multimodal Models](https://mlsanity.com/paper/2506.15564/) (2025)
- [A Survey on Diffusion Language Models](https://mlsanity.com/paper/2508.10875/) (2025)
- [Discrete Diffusion in Large Language and Multimodal Models: A Survey](https://mlsanity.com/paper/2506.13759/) (2025)
- [Lumos-1: On Autoregressive Video Generation with Discrete Diffusion from a Unified Model Perspective](https://mlsanity.com/paper/2507.08801/) (2025)
- [MDNS: Masked Diffusion Neural Sampler via Stochastic Optimal Control](https://mlsanity.com/paper/2508.10684/) (2025)
- [Enhancing Reasoning for Diffusion LLMs via Distribution Matching Policy Optimization](https://mlsanity.com/paper/2510.08233/) (2025)
- [Tuna-2: Pixel Embeddings Beat Vision Encoders for Multimodal Understanding and Generation](https://mlsanity.com/paper/2604.24763/) (2026)
- [OneFlow: Concurrent Mixed-Modal and Interleaved Generation with Edit Flows](https://mlsanity.com/paper/2510.03506/) (2025)
- [Lavida-O: Elastic Large Masked Diffusion Models for Unified Multimodal Understanding and Generation](https://mlsanity.com/paper/2509.19244/) (2025)
- [TR2-D2: Tree Search Guided Trajectory-Aware Fine-Tuning for Discrete Diffusion](https://mlsanity.com/paper/2509.25171/) (2025)
- [SDAR-VL: Stable and Efficient Block-wise Diffusion for Vision-Language Understanding](https://mlsanity.com/paper/2512.14068/) (2025)
- [MM-ACT: Learn from Multimodal Parallel Generation to Act](https://mlsanity.com/paper/2512.00975/) (2025)
- [Does Understanding Inform Generation in Unified Multimodal Models? From Analysis to Path Forward](https://mlsanity.com/paper/2511.20561/) (2025)
- [UEval: A Benchmark for Unified Multimodal Generation](https://mlsanity.com/paper/2601.22155/) (2026)
- [Prism: Efficient Test-Time Scaling via Hierarchical Search and Self-Verification for Discrete Diffusion Language Models](https://mlsanity.com/paper/2602.01842/) (2026)
- [LaViDa-R1: Advancing Reasoning for Unified Multimodal Diffusion Language Models](https://mlsanity.com/paper/2602.14147/) (2026)
- [Evolution of Video Generative Foundations](https://mlsanity.com/paper/2604.06339/) (2026)
- [Soft-Di[M]O: Improving One-Step Discrete Image Generation with Soft Embeddings](https://mlsanity.com/paper/2509.22925/) (2025)
- [Sparse-LaViDa: Sparse Multimodal Discrete Diffusion Language Models](https://mlsanity.com/paper/2512.14008/) (2025)
- [From Masks to Worlds: A Hitchhiker's Guide to World Models](https://mlsanity.com/paper/2510.20668/) (2025)
- [Dynin-Omni: Omnimodal Unified Large Diffusion Language Model](https://mlsanity.com/paper/2604.00007/) (2026)
- [Emotional Support with LLM-based Empathetic Dialogue Generation](https://mlsanity.com/paper/2507.12820/) (2025)
- [RecTok: Reconstruction Distillation along Rectified Flow](https://mlsanity.com/paper/2512.13421/) (2025)
- [UniModel: A Visual-Only Framework for Unified Multimodal Understanding and Generation](https://mlsanity.com/paper/2511.16917/) (2025)
- [Masked Diffusion Models are Secretly Learned-Order Autoregressive Models](https://mlsanity.com/paper/2511.19152/) (2025)
- [FlashAR: Efficient Post-Training Acceleration for Autoregressive Image Generation](https://mlsanity.com/paper/2605.09430/) (2026)
- [Transferability Between Understanding and Generation in Unified Multimodal Models](https://mlsanity.com/paper/2607.04423/) (2026)
- [Does Hearing Help Seeing? Investigating Audio-Video Joint Denoising for Video Generation](https://mlsanity.com/paper/2512.02457/) (2025)
- [dMLLM-TTS: Self-Verified and Efficient Test-Time Scaling for Diffusion Multi-Modal Large Language Models](https://mlsanity.com/paper/2512.19433/) (2025)
- [IMTBench: A Multi-Scenario Cross-Modal Collaborative Evaluation Benchmark for In-Image Machine Translation](https://mlsanity.com/paper/2603.10495/) (2026)

## Abstract (arXiv)

Unified generation models aim to handle diverse tasks across modalities -- such as text generation, image generation, and vision-language reasoning -- within a single architecture and decoding paradigm. Autoregressive unified models suffer from slow inference due to sequential decoding, and non-autoregressive unified models suffer from weak generalization due to limited pretrained backbones. We introduce the second-generation Meissonic: Muddit, a unified discrete diffusion transformer that enables fast and parallel generation across both text and image modalities. Unlike prior unified diffusion models trained from scratch, Muddit integrates strong visual priors from a pretrained text-to-image backbone with a lightweight text decoder, enabling flexible and high-quality multimodal generation under a unified architecture. Empirical results show that Muddit achieves competitive or superior performance compared to significantly larger autoregressive models in both quality and efficiency. The work highlights the potential of purely discrete diffusion, when equipped with strong visual priors, as a scalable and effective backbone for unified generation.