mlsanityData to 14 Oct 2026

Fixed-Point Masked Generative Modeling

arXiv 2605.31215 ↗First posted 29 May 2026cs.LG cs.CVcited by 5

On the map: Efficient LLM inference and serving → Diffusion LLM and MoE inference

Abstract (arXiv)

Masked Generative Models (MGMs) enable parallel decoding and achieve strong performance across modalities, but require full-sequence bidirectional transformers at every step, making training costly and degrading quality under low sampling budgets. Existing work improves efficiency via better samplers or cheaper fixed-depth denoisers, but they still allocate a fixed amount of denoiser computation to each refinement step. We introduce Fixed-Point Masked Generative Models (FP-MGMs), which replace part of the denoiser with a fixed-point solver over shared attention layers to enable adaptive depth with fewer parameters. To make it more effective for masked generation, we first introduce a cross-step consistency loss, which aligns hidden representations at neighboring denoising steps and, second, three-state reuse (3SR) which warm-starts the solver using the previous solution by treating differently unchanged, still-masked, and newly revealed tokens respectively. Together, these components define our complete training-to-inference framework for fixed-point masked generation, CoFRe. We also show that pre-trained MGMs can be converted into FP-MGMs with short fine-tuning, avoiding full retraining. Across modalities, CoFRe improves the quality and cost trade-off. On OpenWebText, CoFRe reduces parameters by 38.8%, training time by 11.5%, and VRAM by 16.9%, while improving generative perplexity from 830.8 to 101.8 at a budget of 96 transformer-block forward passes, compared to MDLM. In ImageNette, CoFRe reduces training time by 48.6% and VRAM by 50.7%, while improving FID in all sample budgets tested. Overall, CoFRe offers a practical framework for cheaper training and stronger low-budget masked generation.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Placed in Diffusion LLM and MoE inference, one of 12 directions in Efficient LLM inference and serving. Builds on 60 of its 71 references that are in the map. First posted on arXiv 29 May 2026.

Builds on

60 of 71 references.

References · 60

  1. 2016Improved Techniques for Training GANscited by 2,970
  2. 2019The Curious Case of Neural Text Degenerationcited by 2,916
  3. 2019Similarity of Neural Network Representations Revisitedcited by 1,755
  4. 2022MaskGIT: Masked Generative Image Transformercited by 1,099
  5. 2023Vision Transformers Need Registerscited by 871
  6. 2024Simple and Effective Masked Diffusion Language Modelscited by 848
  7. 2023Discrete Diffusion Modeling by Estimating the Ratios of the Data Distributioncited by 745
  8. 2024Simplified and Generalized Masked Diffusion for Discrete Datacited by 626
  9. 2019Deep Equilibrium Modelscited by 615
  10. 2025Dream 7B: Diffusion Large Language Modelscited by 544
  11. 2022Phenaki: Variable Length Video Generation From Open Domain Textual Descriptioncited by 484
  12. 2021MAUVE: Measuring the Gap Between Neural Text and Human Text using Divergence Frontierscited by 470
38 more
  1. 2024Your Absorbing Discrete Diffusion Secretly Models the Conditional Distributions of Clean Datacited by 381
  2. 2013One Billion Word Benchmark for Measuring Progress in Statistical Language Modelingcited by 308
  3. 2024Scaling Diffusion Language Models via Adaptation from Autoregressive Modelscited by 274
  4. 2023Looped Transformers as Programmable Computerscited by 237
  5. 2022Continuous diffusion for categorical datacited by 214
  6. 2025Train for the Worst, Plan for the Best: Understanding Token Ordering in Masked Diffusionscited by 214
  7. 2024Masked Diffusion Models are Secretly Time-Agnostic Masked Models and Exploit Inaccurate Categorical Samplingcited by 208
  8. 2024Scaling up Masked Diffusion Models on Textcited by 199
  9. 2025Remasking Discrete Diffusion Models with Inference-Time Scalingcited by 189
  10. 2025The Diffusion Dualitycited by 135
  11. 2025Mixture-of-Recursions: Learning Dynamic Recursive Depths for Adaptive Token-Level Computationcited by 133
  12. 20234M: Massively Multimodal Masked Modelingcited by 120
  13. 2024ImageFolder: Autoregressive Image Generation with Folded Tokenscited by 94
  14. 2021JFB: Jacobian-Free Backpropagation for Implicit Networkscited by 86
  15. 2025Path Planning for Masked Diffusion Model Samplingcited by 80
  16. 2024Relaxed Recursive Transformers: Effective Parameter Sharing with Layer-wise LoRAcited by 78
  17. 2024Beyond Autoregression: Fast LLMs via Self-Distillation Through Timecited by 78
  18. 2024Think While You Generate: Discrete Diffusion with Planned Denoisingcited by 63
  19. 2023One-Step Diffusion Distillation via Deep Equilibrium Modelscited by 61
  20. 2026LoopFormer: Elastic-Depth Looped Transformers for Latent Reasoning via Shortcut Modulationcited by 61
  21. 2026Parcae: Scaling Laws For Stable Looped Language Modelscited by 55
  22. 2025Fast Solvers for Discrete Diffusion Models: Theory and Applications of High-Order Algorithmscited by 47
  23. 2024textitJump Your Steps: Optimizing Sampling Schedule of Discrete Diffusion Modelscited by 40
  24. 2025Plan for Speed: Dilated Scheduling for Masked Diffusion Language Modelscited by 34
  25. 2025Halton Scheduler For Masked Generative Image Transformercited by 32
  26. 2025Learning Unmasking Policies for Diffusion Language Modelscited by 30
  27. 2025Improving Discrete Diffusion Unmasking Policies Beyond Explicit Reference Policiescited by 22
  28. 2025CDLM: Consistency Diffusion Language Models For Faster Samplingcited by 21
  29. 2021Divergence Frontiers for Generative Models: Sample Complexity, Quantization Effects, and Frontier Integralscited by 18
  30. 2024Fixed Point Diffusion Modelscited by 16
  31. 2026Generative Frontiers: Why Evaluation Matters for Diffusion Language Modelscited by 16
  32. 2024SpecMaskGIT: Masked Generative Modeling of Audio Spectrograms for Efficient Audio Synthesis and Beyondcited by 15
  33. 2025Effective and Efficient Masked Image Generation Modelscited by 13
  34. 2024Promises, Outlooks and Challenges of Diffusion Language Modelingcited by 12
  35. 2024XQ-GAN: An Open-source Image Tokenization Framework for Autoregressive Generationcited by 12
  36. 2026SpiralFormer: Looped Transformers Can Learn Hierarchical Dependencies via Multi-Resolution Recursioncited by 12
  37. 2026Beyond Single Tokens: Distilling Discrete Diffusion Models via Discrete MMDcited by 12
  38. 2025Self-Speculative Masked Diffusionscited by 11

50 most cited of 60 All 60 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 5.

5 papers.

  1. 2026Self-conditioned Flow Map Language Models via Fixed-point Flowscited by 14
  2. 2026Loop the Loopies!cited by 8
  3. 2026LiFT: Loop Flow Transformerscited by 1
  4. 2026ALoDLM: Adaptively Looped Diffusion Language Modelscited by 0
  5. 2026Multi-Stream Residual Computation in Diffusion Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Plug-and-Play Context Feature Reuse for Efficient Masked Generation
  2. 2025Masked Conditioning for Deep Generative Models
  3. 2024Unlocking the Capabilities of Masked Generative Models for Image Synthesis via Self-Guidance
  4. 2025MaskFocus: Focusing Policy Optimization on Critical Steps for Masked Image Generation
  5. 2025Beyond Masked and Unmasked: Discrete Diffusion Models via Partial Masking
  6. 2025Accelerating Inference of Masked Image Generators via Reinforcement Learning
  7. 2026Recursive Scaling in Masked Diffusion Models
  8. 2024Efficient Generative Modeling with Residual Vector Quantization-Based Tokens