mlsanityData to 7 Oct 2026

Unified Multimodal Discrete Diffusion

arXiv 2503.20853 ↗First posted 26 Mar 2025cs.CV cs.AI cs.LGcited by 46

On the map: Diffusion models → Image generation, text-to-image and editing

Abstract (arXiv)

Multimodal generative models that can understand and generate across multiple modalities are dominated by autoregressive (AR) approaches, which process tokens sequentially from left to right, or top to bottom. These models jointly handle images, text, video, and audio for various tasks such as image captioning, question answering, and image generation. In this work, we explore discrete diffusion models as a unified generative formulation in the joint text and image domain, building upon their recent success in text generation. Discrete diffusion models offer several advantages over AR models, including improved control over quality versus diversity of generated samples, the ability to perform joint multimodal inpainting (across both text and image domains), and greater controllability in generation through guidance. Leveraging these benefits, we present the first Unified Multimodal Discrete Diffusion (UniDisc) model which is capable of jointly understanding and generating text and images for a variety of downstream tasks. We compare UniDisc to multimodal AR models, performing a scaling analysis and demonstrating that UniDisc outperforms them in terms of both performance and inference-time compute, enhanced controllability, editability, inpainting, and flexible trade-off between inference time and generation quality. Code and additional visualizations are available at https://unidisc.github.io.

Key facts. Cited by 46 papers in the map (data through 7 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 58 of its 68 references that are in the map. First posted on arXiv 26 Mar 2025.

Builds on

58 of 68 references.

Closest ancestors · 6

Same direction, newest first

  1. 2024Playground v2.5: Three Insights towards Enhancing Aesthetic Quality in Text-to-Image Generation27 Feb
  2. 2023SelfEval: Leveraging the discriminative nature of generative models for evaluation17 Nov
  3. 2023GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignment17 Oct
  4. 2023DataComp: In search of the next generation of multimodal datasets27 Apr
  5. 2023Your Diffusion Model is Secretly a Zero-Shot Classifier28 Mar
  6. 2022Unified Discrete Diffusion for Simultaneous Vision-Language Generation27 Nov

Other references · 52

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  5. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  6. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  7. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  8. 2020Scaling Laws for Neural Language Modelscited by 6,411
  9. 2022Classifier-Free Diffusion Guidancecited by 5,717
  10. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  11. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  12. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
38 more
  1. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,551
  2. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,037
  3. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,665
  4. 2015Microsoft COCO Captions: Data Collection and Evaluation Servercited by 1,447
  5. 2019Root Mean Square Layer Normalizationcited by 1,425
  6. 2015Flickr30k Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Modelscited by 1,352
  7. 2021Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Conceptscited by 1,209
  8. 2022MaskGIT: Masked Generative Image Transformercited by 1,082
  9. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,006
  10. 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 904
  11. 2024Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMscited by 822
  12. 2024Simple and Effective Masked Diffusion Language Modelscited by 800
  13. 2024Autoregressive Model Beats Diffusion: Llama for Scalable Image Generationcited by 760
  14. 2023Discrete Diffusion Modeling by Estimating the Ratios of the Data Distributioncited by 712
  15. 2021CogView: Mastering Text-to-Image Generation via Transformerscited by 691
  16. 2023Muse: Text-To-Image Generation via Masked Generative Transformerscited by 640
  17. 2023Language Model Beats Diffusion -- Tokenizer is Key to Visual Generationcited by 633
  18. 2021Perceiver IO: A General Architecture for Structured Inputs & Outputscited by 616
  19. 2024Simplified and Generalized Masked Diffusion for Discrete Datacited by 606
  20. 2021Argmax Flows and Multinomial Diffusion: Learning Categorical Distributionscited by 566
  21. 2022Winoground: Probing Vision and Language Models for Visio-Linguistic Compositionalitycited by 523
  22. 2024Transfusion: Predict the Next Token and Diffuse Images with One Multi-Modal Modelcited by 468
  23. 2022Unified-IO: A Unified Model for Vision, Language, and Multi-Modal Taskscited by 420
  24. 2023Efficient Diffusion Training via Min-SNR Weighting Strategycited by 242
  25. 2024Masked Diffusion Models are Secretly Time-Agnostic Masked Models and Exploit Inaccurate Categorical Samplingcited by 204
  26. 2023Any-to-Any Generation via Composable Diffusioncited by 203
  27. 2024Scaling up Masked Diffusion Models on Textcited by 195
  28. 2023Small-scale proxies for large-scale Transformer training instabilitiescited by 182
  29. 2024Open-MAGVIT2: An Open-Source Project Toward Democratizing Auto-regressive Visual Generationcited by 153
  30. 2024Lumina-mGPT: Illuminate Flexible Photorealistic Text-to-Image Generation with Multimodal Generative Pretrainingcited by 148
  31. 2023Likelihood-Based Diffusion Language Modelscited by 147
  32. 2024Lumina-Next: Making Lumina-T2X Stronger and Faster with Next-DiTcited by 130
  33. 2024What If We Recaption Billions of Web Images with LLaMA-3?cited by 82
  34. 2019CLEVR-Ref+: Diagnosing Visual Reasoning with Referring Expressionscited by 73
  35. 2023Intriguing properties of generative classifierscited by 52
  36. 2024From Pixels to Prose: A Large Dataset of Dense Image Captionscited by 52
  37. 2024Masked Audio Generation using a Single Non-Autoregressive Transformercited by 50
  38. 2024LLM-Inference-Bench: Inference Benchmarking of Large Language Models on AI Acceleratorscited by 31

50 most cited of 52 All 52 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 25; 2026: 21.

46 papers.

Sort:

By area:

  1. 2025Show-o2: Improved Native Unified Multimodal Modelscited by 242
  2. 2025LLaDA-V: Large Language Diffusion Models with Visual Instruction Tuningcited by 145
  3. 2025Unified Multimodal Understanding and Generation Models: Advances, Challenges, and Opportunitiescited by 81
  4. 2025LaViDa: A Large Diffusion Language Model for Multimodal Understandingcited by 81
  5. 2025A Survey on Diffusion Language Modelscited by 75
  6. 2025Discrete Diffusion in Large Language and Multimodal Models: A Surveycited by 61
  7. 2025Diffusion Beats Autoregressive in Data-Constrained Settingscited by 61
  8. 2025Muddit: Liberating Generation Beyond Text-to-Image with a Unified Discrete Diffusion Modelcited by 52
  9. 2025Any-Order Flexible Length Masked Diffusioncited by 48
  10. 2025FUDOKI: Discrete Flow-based Unified Understanding and Generation via Kinetic-Optimal Velocitiescited by 46
  11. 2025Learning Unmasking Policies for Diffusion Language Modelscited by 29
  12. 2025Lumos-1: On Autoregressive Video Generation with Discrete Diffusion from a Unified Model Perspectivecited by 22
34 more
  1. 2025Diffuse Everything: Multimodal Diffusion Models on Arbitrary State Spacescited by 19
  2. 2025OneFlow: Concurrent Mixed-Modal and Interleaved Generation with Edit Flowscited by 17
  3. 2025AsyncVLA: Asynchronous Flow Matching for Vision-Language-Action Modelscited by 16
  4. 2025From Denoising to Refining: A Corrective Framework for Vision-Language Diffusion Modelcited by 9
  5. 2025Pulp Motion: Framing-aware multimodal camera and human motion generationcited by 8
  6. 2026LLaDA-o: An Effective and Length-Adaptive Omni Diffusion Modelcited by 8
  7. 2025Discrete Guidance Matching: Exact Guidance for Discrete Flow Matchingcited by 6
  8. 2026Context Tokens are Anchors: Understanding the Repetition Curse in dMLLMs from an Information Flow Perspectivecited by 6
  9. 2026UReason: Benchmarking Reasoning-to-Generation Alignment in Unified Multimodal Modelscited by 6
  10. 2026LaViDa-R1: Advancing Reasoning for Unified Multimodal Diffusion Language Modelscited by 6
  11. 2026The Design Space of Tri-Modal Masked Diffusion Modelscited by 6
  12. 2025From Masks to Worlds: A Hitchhiker's Guide to World Modelscited by 5
  13. 2025ReDDiT: Rehashing Noise for Discrete Visual Generationcited by 3
  14. 2025FAPE-IR: Frequency-Aware Planning and Execution Framework for All-in-One Image Restorationcited by 3
  15. 2026Breaking Dual Bottlenecks: Evolving Unified Multimodal Models into Self-Adaptive Interleaved Visual Reasonerscited by 3
  16. 2026IMTBench: A Multi-Scenario Cross-Modal Collaborative Evaluation Benchmark for In-Image Machine Translationcited by 2
  17. 2026DVD: Discrete Voxel Diffusion for 3D Generation and Editingcited by 2
  18. 2026Mask-Aware Policy Gradients for Diffusion Language Modelscited by 2
  19. 2025Discrete Diffusion Models with MLLMs for Unified Medical Multimodal Generationcited by 1
  20. 2025Masked Diffusion Captioning for Visual Feature Learningcited by 1
  21. 2025Generative AI Meets 6G and Beyond: Diffusion Models for Semantic Communicationscited by 1
  22. 2025FlowBind: Efficient Any-to-Any Generation with Bidirectional Flowscited by 1
  23. 2026Best of Both Worlds: Multimodal Reasoning and Generation via Unified Discrete Flow Matchingcited by 1
  24. 2026LADR: Locality-Aware Dynamic Rescue for Efficient Text-to-Image Generation with Diffusion Large Language Modelscited by 1
  25. 2026Know3D: Prompting 3D Generation with Knowledge from Vision-Language Modelscited by 1
  26. 2026TBD-VLA: Temporal Block Diffusion Vision Language Action Modelcited by 1
  27. 2026UniTranslator: A Unified Multi-modal Framework for End-to-end In-Image Machine Translationcited by 1
  28. 2026AsyncPatch Diffusion: spatially-flexible image generationcited by 0
  29. 2026Efficient Reinforcement for Visual-Textual Thinking with Discrete Diffusion Modelcited by 0
  30. 2026Context-Aware Cluster Decoding: Semantic Anchor-Driven Coherence in dMLLMscited by 0
  31. 2026Discrete Diffusion Bridges for Spatiotemporally Aligned Image Translation and Generationcited by 0
  32. 2026Dynin-Robotics: Omnimodal Unified Diffusion Vision-Language-Action Modelcited by 0
  33. 2026Multimodal Flow: Unified Flow Modeling of Language and Vision in Embedding Spacescited by 0
  34. 2026Omni-Diffusion-Distill: Few-Step Distillation of Unified Multimodal Diffusion Large Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Dual Diffusion for Unified Image Generation and Understanding
  2. 2024ACDC: Autoregressive Coherent Multimodal Generation using Diffusion Correction
  3. 2025Proxy-Tuning: Tailoring Multimodal Autoregressive Models for Subject-Driven Image Generation
  4. 2023Efficient Multimodal Diffusion Models Using Joint Data Infilling with Partially Shared U-Net
  5. 2026Omni-Diffusion: Unified Multimodal Understanding and Generation with Masked Discrete Diffusion
  6. 2024UNIMO-G: Unified Image Generation through Multimodal Conditional Diffusion
  7. 2025UniCMs: A Unified Consistency Model For Efficient Multimodal Generation and Understanding
  8. 2026UniCanvas: A Diffusion-base Unified Model for Text-in-Image Joint Generation