mlsanityData to 7 Oct 2026

Dimple: Discrete Diffusion Multimodal Large Language Model with Parallel Decoding

arXiv 2505.16990 ↗First posted 22 May 2025cs.CVcited by 101

On the map: Multimodal LLMs / vision-language models → Image editing and quality assessment

also in 2 more
Abstract (arXiv)

In this work, we propose Dimple, the first Discrete Diffusion Multimodal Large Language Model (DMLLM). We observe that training with a purely discrete diffusion approach leads to significant training instability, suboptimal performance, and severe length bias issues. To address these challenges, we design a novel training paradigm that combines an initial autoregressive phase with a subsequent diffusion phase. This approach yields the Dimple-7B model, trained on the same dataset and using a similar training pipeline as LLaVA-NEXT. Dimple-7B ultimately surpasses LLaVA-NEXT in performance by 3.9%, demonstrating that DMLLM can achieve performance comparable to that of autoregressive models. To improve inference efficiency, we propose a decoding strategy termed confident decoding, which dynamically adjusts the number of tokens generated at each step, significantly reducing the number of generation iterations. In autoregressive models, the number of forward iterations during generation equals the response length. With confident decoding, however, the number of iterations needed by Dimple is even only fracresponse length3. We also re-implement the prefilling technique in autoregressive models and demonstrate that it does not significantly impact performance on most benchmark evaluations, while offering a speedup of 1.5x to 7x. Additionally, we explore Dimple's capability to precisely control its response using structure priors. These priors enable structured responses in a manner distinct from instruction-based or chain-of-thought prompting, and allow fine-grained control over response format and length, which is difficult to achieve in autoregressive models. Overall, this work validates the feasibility and advantages of DMLLM and enhances its inference efficiency and controllability. Code and models are available at https://github.com/yu-rp/Dimple.

Key facts. Cited by 101 papers in the map (data through 7 Oct 2026). Placed in Image editing and quality assessment, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 40 of its 63 references that are in the map. First posted on arXiv 22 May 2025.

Builds on

40 of 63 references.

References · 40

  1. 2023Visual Instruction Tuningcited by 7,792
  2. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  3. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
  4. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,179
  5. 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,105
  6. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,044
  7. 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answeringcited by 1,881
  8. 2019Towards VQA Models That Can Readcited by 1,696
  9. 2023Evaluating Object Hallucination in Large Vision-Language Modelscited by 1,696
  10. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,665
  11. 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,624
  12. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,582
28 more
  1. 2023MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilitiescited by 1,221
  2. 2024How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suitescited by 1,033
  3. 2016A Diagram Is Worth A Dozen Imagescited by 793
  4. 2023Discrete Diffusion Modeling by Estimating the Ratios of the Data Distributioncited by 712
  5. 2024Autoregressive Image Generation without Vector Quantizationcited by 644
  6. 2023Muse: Text-To-Image Generation via Masked Generative Transformerscited by 640
  7. 2024Simplified and Generalized Masked Diffusion for Discrete Datacited by 606
  8. 2021Argmax Flows and Multinomial Diffusion: Learning Categorical Distributionscited by 566
  9. 2023OCRBench: On the Hidden Mystery of OCR in Large Multimodal Modelscited by 466
  10. 2024VLMEvalKit: An Open-Source Toolkit for Evaluating Large Multi-Modality Modelscited by 443
  11. 2022A Continuous Time Framework for Discrete Denoising Modelscited by 437
  12. 2024Generative Flows on Discrete State-Spaces: Enabling Multimodal Flows with Applications to Protein Co-Designcited by 294
  13. 2024Scaling Diffusion Language Models via Adaptation from Autoregressive Modelscited by 268
  14. 2022MAGE: MAsked Generative Encoder to Unify Representation Learning and Image Synthesiscited by 222
  15. 2022SSD-LM: Semi-autoregressive Simplex-based Diffusion Language Model for Text Generation and Modular Controlcited by 191
  16. 2022Score-based Continuous-time Discrete Diffusion Modelscited by 167
  17. 2023A Reparameterized Discrete Diffusion Model for Text Generationcited by 152
  18. 2022Concrete Score Matching: Generalized Score Matching for Discrete Datacited by 144
  19. 2024Beyond Autoregression: Discrete Diffusion for Complex Reasoning and Planningcited by 135
  20. 2024MAmmoTH-VL: Eliciting Multimodal Reasoning with Instruction Tuning at Scalecited by 125
  21. 2022Self-conditioned Embedding Diffusion for Text Generationcited by 114
  22. 2024Mono-InternVL: Pushing the Boundaries of Monolithic Multimodal Large Language Models with Endogenous Visual Pre-trainingcited by 82
  23. 2022SeqDiffuSeq: Text Diffusion with Encoder-Decoder Transformerscited by 63
  24. 2024Masked Audio Generation using a Single Non-Autoregressive Transformercited by 50
  25. 2023Diffusion Models for Non-autoregressive Text Generation: A Surveycited by 43
  26. 2024Joint Vision-Language Social Bias Removal for CLIPcited by 17
  27. 2024LLaVA-Chef: A Multi-modal Generative Model for Food Recipescited by 14
  28. 2022CLIP-Diffusion-LM: Apply Diffusion Model on Image Captioningcited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 47; 2026: 54.

101 papers.

Sort:

By area:

  1. 2025Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decodingcited by 404
  2. 2025DiffuCoder: Understanding and Improving Masked Diffusion Models for Code Generationcited by 184
  3. 2025Fast-dLLM v2: Efficient Block-Diffusion LLMcited by 128
  4. 2025Lumina-DiMOO: An Omni Diffusion Large Language Model for Multi-Modal Generation and Understandingcited by 83
  5. 2025A Survey on Diffusion Language Modelscited by 75
  6. 2025Discrete Diffusion in Large Language and Multimodal Models: A Surveycited by 61
  7. 2025dParallel: Learnable Parallel Decoding for dLLMscited by 59
  8. 2025LLaDA-MoE: A Sparse MoE Diffusion Language Modelcited by 54
  9. 2025KLASS: KL-Guided Fast Inference in Masked Diffusion Modelscited by 44
  10. 2025LongLLaDA: Unlocking Long Context Capabilities in Diffusion LLMscited by 38
  11. 2025Beyond Fixed: Training-Free Variable-Length Denoising for Diffusion Large Language Modelscited by 32
  12. 2025Attention Is All You Need for KV Cache in Diffusion LLMscited by 32
38 more
  1. 2025AdaBlock-dLLM: Semantic-Aware Diffusion LLM Inference via Adaptive Block Sizecited by 30
  2. 2025Principled RL for Diffusion LLMs Emerges from a Sequence-Level Perspectivecited by 30
  3. 2025Learning Unmasking Policies for Diffusion Language Modelscited by 29
  4. 2025A Survey on Parallel Text Generation: From Parallel Decoding to Diffusion Language Modelscited by 27
  5. 2025LaDiR: Latent Diffusion Enhances LLMs for Text Reasoningcited by 25
  6. 2025Set Block Decoding is a Language Model Inference Acceleratorcited by 22
  7. 2025From Bits to Rounds: Parallel Decoding with Exploration for Diffusion Language Modelscited by 22
  8. 2025Empirical Analysis of Decoding Biases in Masked Diffusion Modelscited by 19
  9. 2025Free Draft-and-Verification: Toward Lossless Parallel Decoding for Diffusion Large Language Modelscited by 19
  10. 2026Nemotron-Labs-Diffusion: A Tri-Mode Language Model Unifying Autoregressive, Diffusion, and Self-Speculation Decodingcited by 19
  11. 2025Dream-VL & Dream-VLA: Open Vision-Language and Vision-Language-Action Models with Diffusion Language Model Backbonecited by 18
  12. 2026Confidence-Based Decoding is Provably Efficient for Diffusion Language Modelscited by 17
  13. 2025WAM-Flow: Parallel Coarse-to-Fine Motion Planning via Discrete Flow Matching for Autonomous Drivingcited by 15
  14. 2025Lavida-O: Elastic Large Masked Diffusion Models for Unified Multimodal Understanding and Generationcited by 14
  15. 2025dVLM-AD: Enhance Diffusion Vision-Language-Model for Driving via Controllable Reasoningcited by 14
  16. 2026DMax: Aggressive Parallel Decoding for dLLMscited by 14
  17. 2025Saber: An Efficient Sampling with Adaptive Acceleration and Backtracking Enhanced Remasking for Diffusion Language Modelcited by 13
  18. 2025SDAR-VL: Stable and Efficient Block-wise Diffusion for Vision-Language Understandingcited by 12
  19. 2026Continuous Latent Diffusion Language Modelcited by 12
  20. 2025CreditDecoding: Accelerating Parallel Decoding in Diffusion Large Language Models with Trace Creditcited by 11
  21. 2025DiffusionVL: Translating Any Autoregressive Models into Diffusion Vision Language Modelscited by 10
  22. 2026MMaDA-VLA: Large Diffusion Vision-Language-Action Model with Unified Multi-Modal Instruction and Generationcited by 10
  23. 2025A2D: Any-Order, Any-Step Safety Alignment for Diffusion Language Modelscited by 9
  24. 2025Latent Refinement Decoding: Enhancing Diffusion-Based Language Models by Refining Belief Statescited by 9
  25. 2025From Denoising to Refining: A Corrective Framework for Vision-Language Diffusion Modelcited by 9
  26. 2025Beyond Confidence: Adaptive and Coherent Decoding for Diffusion Language Modelscited by 9
  27. 2026Adaptation to Intrinsic Dependence in Diffusion Language Modelscited by 9
  28. 2026Omni-Diffusion: Unified Multimodal Understanding and Generation with Masked Discrete Diffusioncited by 9
  29. 2026CD4LM: Consistency Distillation and aDaptive Decoding for Diffusion Language Modelscited by 8
  30. 2026LLaDA-o: An Effective and Length-Adaptive Omni Diffusion Modelcited by 8
  31. 2026Fast-dVLM: Efficient Block-Diffusion VLM via Direct Conversion from Autoregressive VLMcited by 8
  32. 2025Discrete Diffusion for Generative Modeling of Text-Aligned Speech Tokenscited by 7
  33. 2025RFG: Test-Time Scaling for Diffusion Large Language Model Reasoning with Reward-Free Guidancecited by 7
  34. 2026MinerU-Diffusion: Rethinking Document OCR as Inverse Rendering via Diffusion Decodingcited by 7
  35. 2026When to Commit? Towards Variable-Size Self-Contained Blocks for Discrete Diffusion Language Modelscited by 7
  36. 2025Sparse-LaViDa: Sparse Multimodal Discrete Diffusion Language Modelscited by 6
  37. 2026Context Tokens are Anchors: Understanding the Repetition Curse in dMLLMs from an Information Flow Perspectivecited by 6
  38. 2026LaViDa-R1: Advancing Reasoning for Unified Multimodal Diffusion Language Modelscited by 6

50 most cited of 101 All 101 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026d3LLM: Ultra-Fast Diffusion LLM using Pseudo-Trajectory Distillation
  2. 2025Learning to Parallel: Accelerating Diffusion Large Language Models via Learnable Parallel Decoding
  3. 2026Early-Bird Decoding: Accelerating Diffusion LLMs with Learnable Block Sizes and Parallel Sampling
  4. 2025LLaDA2.0: Scaling Up Diffusion Language Models to 100B
  5. 2025Fast-Decoding Diffusion Language Models via Progress-Aware Confidence Schedules
  6. 2026Locally Coherent Parallel Decoding in Diffusion Language Models
  7. 2025Diffusion LLMs Can Do Faster-Than-AR Inference via Discrete Diffusion Forcing
  8. 2026Understanding Evaluation Illusion in Diffusion Large Language Models