mlsanityData to 7 Oct 2026

Partition Generative Modeling: Masked Modeling Without Masks

arXiv 2505.18883 ↗First posted 24 May 2025cs.LGcited by 6

Not on the map. Why?

Abstract (arXiv)

Masked generative models (MGMs) can generate tokens in parallel and in any order, unlike autoregressive models (ARMs), which decode one token at a time, left-to-right. However, MGMs process the full-length sequence at every sampling step, including mask tokens that carry no information. In contrast, ARMs process only the previously generated tokens. We introduce ``Partition Generative Models'' (PGMs), which replace masking with partitioning. Tokens are split into two groups that cannot attend to each other, and the model learns to predict each group conditioned on the other, eliminating mask tokens entirely. Because the groups do not interact, PGMs can process only the clean tokens during sampling, like ARMs, while retaining parallel, any-order generation, like MGMs. On OpenWebText, PGMs achieve 5-5.5× higher throughput than MDLM while producing samples with lower Generative Perplexity. On ImageNet, PGMs reach comparable FID to MaskGIT with a 7.5× throughput improvement. With twice as many steps, the FID improves to 4.56 while remaining 3.9× faster than MGMs. Finally, PGMs remain compatible with existing MGM samplers and distillation methods.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 45 of its 66 references that are in the map. First posted on arXiv 24 May 2025.

Builds on

45 of 66 references.

References · 45

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  5. 2022Scalable Diffusion Models with Transformerscited by 5,931
  6. 2016Layer Normalizationcited by 5,579
  7. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  8. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  9. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  10. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  11. 2016Improved Techniques for Training GANscited by 2,938
  12. 2019The Curious Case of Neural Text Degenerationcited by 2,881
33 more
  1. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,801
  2. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,928
  3. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,665
  4. 2021Variational Diffusion Modelscited by 1,145
  5. 2024Large Language Monkeys: Scaling Inference Compute with Repeated Samplingcited by 946
  6. 2023Stochastic Interpolants: A Unifying Framework for Flows and Diffusionscited by 897
  7. 2024Simple and Effective Masked Diffusion Language Modelscited by 800
  8. 2023Discrete Diffusion Modeling by Estimating the Ratios of the Data Distributioncited by 712
  9. 2024Simplified and Generalized Masked Diffusion for Discrete Datacited by 606
  10. 2022Efficiently Scaling Transformer Inferencecited by 529
  11. 2022Phenaki: Variable Length Video Generation From Open Domain Textual Descriptioncited by 475
  12. 2022A Continuous Time Framework for Discrete Denoising Modelscited by 437
  13. 2025Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Modelscited by 385
  14. 2024Your Absorbing Discrete Diffusion Secretly Models the Conditional Distributions of Clean Datacited by 371
  15. 2013One Billion Word Benchmark for Measuring Progress in Statistical Language Modelingcited by 302
  16. 2024Generative Flows on Discrete State-Spaces: Enabling Multimodal Flows with Applications to Protein Co-Designcited by 294
  17. 2023Think before you speak: Training Language Models With Pause Tokenscited by 254
  18. 2024Masked Diffusion Models are Secretly Time-Agnostic Masked Models and Exploit Inaccurate Categorical Samplingcited by 204
  19. 2022Continuous diffusion for categorical datacited by 202
  20. 2024Scaling up Masked Diffusion Models on Textcited by 195
  21. 2024Flex Attention: A Programming Model for Generating Optimized Attention Kernelscited by 180
  22. 2024Simple Guidance Mechanisms for Discrete Diffusion Modelscited by 148
  23. 2024Beyond Autoregression: Fast LLMs via Self-Distillation Through Timecited by 73
  24. 2025Generalized Interpolating Discrete Diffusioncited by 68
  25. 2022Training and Inference on Any-Order Autoregressive Models the Right Waycited by 64
  26. 2024Distillation of Discrete Diffusion through Dimensional Correlationscited by 46
  27. 2025Halton Scheduler For Masked Generative Image Transformercited by 30
  28. 2025Target Concrete Score Matching: A Holistic Framework for Discrete Diffusioncited by 25
  29. 2025Distilling Diversity and Control in Diffusion Modelscited by 21
  30. 2024SpecMaskGIT: Masked Generative Modeling of Audio Spectrograms for Efficient Audio Synthesis and Beyondcited by 15
  31. 2025Reviving Any-Subset Autoregressive Models with Principled Parallel Sampling and Speculative Decodingcited by 14
  32. 2024Unified Discrete Diffusion for Categorical Datacited by 11
  33. 2024Promises, Outlooks and Challenges of Diffusion Language Modelingcited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 4.

6 papers.

  1. 2026The Diffusion Duality, Chapter II: Psi-Samplerscited by 12
  2. 2025Loopholing Discrete Diffusion: Deterministic Bypass of the Sampling Wallcited by 6
  3. 2025Efficient Autoregressive Inference for Transformer Probabilistic Modelscited by 4
  4. 2026Fixed-Point Masked Generative Modelingcited by 2
  5. 2026Extracting Training Data from Diffusion Language Models via Infillingcited by 0
  6. 2026Looped Diffusion Language Modelscited by 0