mlsanityData to 7 Oct 2026

MDTv2: Masked Diffusion Transformer is a Strong Image Synthesizer

arXiv 2303.14389 ↗First posted 25 Mar 2023cs.CVcited by 260

On the map: LLM reasoning → Diffusion reasoning and test-time scaling

also in 1 more
Abstract (arXiv)

Despite its success in image synthesis, we observe that diffusion probabilistic models (DPMs) often lack contextual reasoning ability to learn the relations among object parts in an image, leading to a slow learning process. To solve this issue, we propose a Masked Diffusion Transformer (MDT) that introduces a mask latent modeling scheme to explicitly enhance the DPMs' ability to contextual relation learning among object semantic parts in an image. During training, MDT operates in the latent space to mask certain tokens. Then, an asymmetric diffusion transformer is designed to predict masked tokens from unmasked ones while maintaining the diffusion generation process. Our MDT can reconstruct the full information of an image from its incomplete contextual input, thus enabling it to learn the associated relations among image tokens. We further improve MDT with a more efficient macro network structure and training strategy, named MDTv2. Experimental results show that MDTv2 achieves superior image synthesis performance, e.g., a new SOTA FID score of 1.58 on the ImageNet dataset, and has more than 10x faster learning speed than the previous SOTA DiT. The source code is released at https://github.com/sail-sg/MDT.

Key facts. Cited by 260 papers in the map (data through 7 Oct 2026). Placed in Diffusion reasoning and test-time scaling, one of 17 directions in LLM reasoning. Builds on 52 of its 65 references that are in the map. First posted on arXiv 25 Mar 2023.

Builds on

52 of 65 references.

References · 52

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  5. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  6. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  7. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  8. 2020Denoising Diffusion Implicit Modelscited by 9,655
  9. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  10. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  11. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  12. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
38 more
  1. 2022Scalable Diffusion Models with Transformerscited by 5,931
  2. 2022Classifier-Free Diffusion Guidancecited by 5,717
  3. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  4. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  5. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,736
  6. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  7. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 2,994
  8. 2016Improved Techniques for Training GANscited by 2,938
  9. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
  10. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,360
  11. 2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Stepscited by 1,961
  12. 2018Group Normalizationcited by 1,514
  13. 2019Generating Diverse High-Fidelity Images with VQ-VAE-2cited by 1,376
  14. 2021Cascaded Diffusion Models for High Fidelity Image Generationcited by 1,155
  15. 2021Variational Diffusion Modelscited by 1,145
  16. 2022MaskGIT: Masked Generative Image Transformercited by 1,082
  17. 2020Improved Techniques for Training Score-Based Generative Modelscited by 1,036
  18. 2019Improved Precision and Recall Metric for Assessing Generative Modelscited by 925
  19. 2021iBOT: Image BERT Pre-Training with Online Tokenizercited by 891
  20. 2016Conditional Image Generation with PixelCNN Decoderscited by 859
  21. 2021Vector Quantized Diffusion Model for Text-to-Image Synthesiscited by 773
  22. 2021Maximum Likelihood Training of Score-Based Diffusion Modelscited by 670
  23. 2023Muse: Text-To-Image Generation via Masked Generative Transformerscited by 640
  24. 2021Score-based Generative Modeling in Latent Spacecited by 634
  25. 2022StyleGAN-XL: Scaling StyleGAN to Large Diverse Datasetscited by 489
  26. 2017PixelCNN++: Improving the PixelCNN with Discretized Logistic Mixture Likelihood and Other Modificationscited by 415
  27. 2021Generating Images with Sparse Representationscited by 262
  28. 2023Efficient Diffusion Training via Min-SNR Weighting Strategycited by 242
  29. 2021Attention Mechanisms in Computer Vision: A Surveycited by 232
  30. 2021Score-Based Generative Modeling with Critically-Damped Langevin Diffusioncited by 230
  31. 2022Adan: Adaptive Nesterov Momentum Algorithm for Faster Optimizing Deep Modelscited by 172
  32. 2021Diff-TTS: A Denoising Diffusion Model for Text-to-Speechcited by 156
  33. 2021Large-scale Unsupervised Semantic Segmentationcited by 98
  34. 2021Soft Truncation: A Universal Training Technique of Score-based Diffusion Model for High Precision Score Estimationcited by 95
  35. 2022Estimating the Optimal Covariance with Imperfect Mean in Diffusion Probabilistic Modelscited by 83
  36. 2022Your ViT is Secretly a Hybrid Discriminative-Generative Diffusion Modelcited by 58
  37. 2021M6-UFC: Unifying Multi-Modal Controls for Conditional Image Synthesis via Non-Autoregressive Generative Transformerscited by 36
  38. 2022Rethinking the Objectives of Vector-Quantized Tokenizers for Image Synthesiscited by 31

50 most cited of 52 All 52 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 28; 2024: 85; 2025: 90; 2026: 57.

260 papers.

Sort:

By area:

  1. 2024Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Predictioncited by 1,005
  2. 2024SiT: Exploring Flow and Diffusion-based Generative Models with Scalable Interpolant Transformerscited by 707
  3. 2024Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Thinkcited by 666
  4. 2024Autoregressive Image Generation without Vector Quantizationcited by 644
  5. 2024Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusioncited by 634
  6. 2023Language Model Beats Diffusion -- Tokenizer is Key to Visual Generationcited by 633
  7. 2024Sora: A Review on Background, Technology, Limitations, and Opportunities of Large Vision Modelscited by 467
  8. 2025Reconstruction vs. Generation: Taming Optimization Dilemma in Latent Diffusion Modelscited by 376
  9. 2024PixArt-Sigma: Weak-to-Strong Training of Diffusion Transformer for 4K Text-to-Image Generationcited by 326
  10. 2023Photorealistic Video Generation with Diffusion Modelscited by 304
  11. 2024An Image is Worth 32 Tokens for Reconstruction and Generationcited by 295
  12. 2024Guiding a Diffusion Model with a Bad Version of Itselfcited by 286
38 more
  1. 2024Applying Guidance in a Limited Interval Improves Sample and Distribution Quality in Diffusion Modelscited by 240
  2. 2025REPA-E: Unlocking VAE for End-to-End Tuning with Latent Diffusion Transformerscited by 212
  3. 2023Fast Training of Diffusion Models with Masked Transformerscited by 172
  4. 2024Learning-to-Cache: Accelerating Diffusion Transformer via Layer Cachingcited by 138
  5. 2024Randomized Autoregressive Visual Generationcited by 125
  6. 2023CADS: Unleashing the Diversity of Diffusion Models through Condition-Annealed Samplingcited by 124
  7. 2024Scalable High-Resolution Pixel-Space Image Synthesis with Hourglass Diffusion Transformerscited by 116
  8. 2023DiffiT: Diffusion Vision Transformers for Image Generationcited by 104
  9. 2025DDT: Decoupled Diffusion Transformercited by 100
  10. 2024MaskBit: Embedding-free Image Generation via Bit Tokenscited by 96
  11. 2023Return of Unconditional Generation: A Self-supervised Representation Generation Methodcited by 95
  12. 2025Representation Entanglement for Generation: Training Diffusion Transformers Is Much Easier Than You Thinkcited by 95
  13. 2024FiT: Flexible Vision Transformer for Diffusion Modelcited by 84
  14. 2024RandAR: Decoder-only Autoregressive Visual Generation in Random Orderscited by 82
  15. 2023GenTron: Diffusion Transformers for Image and Video Generationcited by 80
  16. 2023Relay Diffusion: Unifying diffusion process across resolutions for image synthesiscited by 78
  17. 2024PTQ4DiT: Post-training Quantization for Diffusion Transformerscited by 77
  18. 2023Diffusion Models Without Attentioncited by 75
  19. 2025Beyond Next-Token: Next-X Prediction for Autoregressive Visual Generationcited by 71
  20. 2024Analysis of Classifier-Free Guidance Weight Schedulerscited by 64
  21. 2024FasterDiT: Towards Faster Diffusion Transformers Training without Architecture Modificationcited by 64
  22. 2023On the Design Fundamentals of Diffusion Models: A Surveycited by 59
  23. 2024SoftVQ-VAE: Efficient 1-Dimensional Continuous Tokenizercited by 55
  24. 2024Sora as a World Model? A Complete Survey on Text-to-Video Generationcited by 53
  25. 2025Token Reduction Should Go Beyond Efficiency in Generative Models -- From Vision, Language to Multimodalitycited by 49
  26. 2024SD-DiT: Unleashing the Power of Self-supervised Discrimination in Diffusion Transformercited by 47
  27. 2023Detecting Images Generated by Deep Diffusion Models using their Local Intrinsic Dimensionalitycited by 44
  28. 2025Boosting Generative Image Modeling via Joint Image-Feature Synthesiscited by 42
  29. 2023GSURE-Based Diffusion Model Training with Corrupted Datacited by 40
  30. 2025VisualCloze: A Universal Image Generation Framework via Visual In-Context Learningcited by 39
  31. 2024Diffusion-Based Neural Network Weights Generationcited by 38
  32. 2024Improved Noise Schedule for Diffusion Trainingcited by 38
  33. 2024On the Scalability of Diffusion-based Text-to-Image Generationcited by 36
  34. 2024Alleviating Distortion in Image Generation via Multi-Resolution Diffusion Models and Time-Dependent Layer Normalizationcited by 36
  35. 2024Language-Guided Image Tokenization for Generationcited by 36
  36. 2025Reangle-A-Video: 4D Video Generation as Video-to-Video Translationcited by 35
  37. 2025AlignTok: Aligning Visual Foundation Encoders to Tokenizers for Diffusion Modelscited by 35
  38. 2024DiffuseHigh: Training-free Progressive High-Resolution Image Synthesis through Structure Guidancecited by 33

50 most cited of 260 All 260 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Masked Autoencoders Are Effective Tokenizers for Diffusion Models
  2. 2022Exploring Transformer Backbones for Image Diffusion Models
  3. 2026SDiT: Semantic Region-Adaptive for Diffusion Transformers
  4. 2023Masked Diffusion as Self-supervised Representation Learner
  5. 2026Nemotron-Labs-Diffusion-Image: Advancing Masked Discrete Diffusion for High-Resolution Image Synthesis
  6. 2024Bag of Design Choices for Inference of High-Resolution Masked Generative Transformer
  7. 2023LMD: Faster Image Reconstruction with Latent Masking Diffusion
  8. 2024Simplified and Generalized Masked Diffusion for Discrete Data