mlsanityData to 14 Oct 2026

Any-Order Flexible Length Masked Diffusion

arXiv 2509.01025 ↗First posted 31 Aug 2025cs.LGcited by 51

On the map: Diffusion models → Discrete and text diffusion

Abstract (arXiv)

Masked diffusion models (MDMs) have recently emerged as a promising alternative to autoregressive models over discrete domains. MDMs generate sequences in an any-order, parallel fashion, enabling fast inference and strong performance on non-causal tasks. However, a crucial limitation is that they do not support token insertions and are thus limited to fixed-length generations. To this end, we introduce Flexible Masked Diffusion Models (FlexMDMs), a discrete diffusion paradigm that simultaneously can model sequences of flexible length while provably retaining MDMs' flexibility of any-order inference. Grounded in an extension of the stochastic interpolant framework, FlexMDMs generate sequences by inserting mask tokens and unmasking them. Empirically, we show that FlexMDMs match MDMs in perplexity while modeling length statistics with much higher fidelity. On a synthetic maze planning task, they achieve ≈ 60 % higher success rate than MDM baselines. Finally, we show pretrained MDMs can easily be retrofitted into FlexMDMs: on 16 H100s, it takes only three days to fine-tune LLaDA-8B into a FlexMDM, achieving superior performance on math (GSM8K, 58% → 67%) and code infilling performance (52% → 65%).

Key facts. Cited by 51 papers in the map (data through 14 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. Builds on 56 of its 60 references that are in the map. First posted on arXiv 31 Aug 2025.

Builds on

56 of 60 references.

Closest ancestors · 17

Same direction, newest first

  1. 2025Reinforced Context Order Recovery for Adaptive Reasoning and Planning18 Aug
  2. 2025Seed Diffusion: A Large-Scale Diffusion Language Model with High-Speed Inference4 Aug
  3. 2025DiffuCoder: Understanding and Improving Masked Diffusion Models for Code Generation25 Jun
  4. 2025Accelerated Sampling from Masked Diffusion Models via Entropy Bounded Unmasking30 May
  5. 2025Anchored Diffusion Language Model24 May
  6. 2025dKV-Cache: The Cache for Diffusion Language Models21 May
  7. 2025Large Language Diffusion Models14 Feb
  8. 2025Train for the Worst, Plan for the Best: Understanding Token Ordering in Masked Diffusions10 Feb
  9. 2025Path Planning for Masked Diffusion Model Sampling5 Feb
  10. 2024Scaling up Masked Diffusion Models on Text24 Oct
  11. 2024Scaling Diffusion Language Models via Adaptation from Autoregressive Models23 Oct
  12. 2024Beyond Autoregression: Discrete Diffusion for Complex Reasoning and Planning18 Oct
5 more
  1. 2024Masked Diffusion Models are Secretly Time-Agnostic Masked Models and Exploit Inaccurate Categorical Sampling4 Sep
  2. 2024Simple and Effective Masked Diffusion Language Models11 Jun
  3. 2024Simplified and Generalized Masked Diffusion for Discrete Data6 Jun
  4. 2024Your Absorbing Discrete Diffusion Secretly Models the Conditional Distributions of Clean Data6 Jun
  5. 2023Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution25 Oct

Other references · 39

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  3. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  4. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  5. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  6. 2022Scalable Diffusion Models with Transformerscited by 6,054
  7. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,700
  8. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
  9. 2024Qwen2 Technical Reportcited by 2,120
  10. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,719
  11. 2022MaskGIT: Masked Generative Image Transformercited by 1,099
  12. 2022Planning with Diffusion for Flexible Behavior Synthesiscited by 1,061
27 more
  1. 2022Building Normalizing Flows with Stochastic Interpolantscited by 995
  2. 2023Stochastic Interpolants: A Unifying Framework for Flows and Diffusionscited by 939
  3. 2024DeepSeek LLM: Scaling Open-Source Language Models with Longtermismcited by 642
  4. 2021Argmax Flows and Multinomial Diffusion: Learning Categorical Distributionscited by 574
  5. 2022A Continuous Time Framework for Discrete Denoising Modelscited by 451
  6. 2025Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decodingcited by 420
  7. 2024Discrete Flow Matchingcited by 317
  8. 2024Generative Flows on Discrete State-Spaces: Enabling Multimodal Flows with Applications to Protein Co-Designcited by 305
  9. 2022Efficient Training of Language Models to Fill in the Middlecited by 255
  10. 2025Mercury: Ultra-Fast Language Models Based on Diffusioncited by 235
  11. 2025d1: Scaling Reasoning in Diffusion Large Language Models via Reinforcement Learningcited by 186
  12. 2024Flex Attention: A Programming Model for Generating Optimized Attention Kernelscited by 185
  13. 2022An optimal control perspective on diffusion-based generative modelingcited by 147
  14. 2024OpenCoder: The Open Cookbook for Top-Tier Code Large Language Modelscited by 130
  15. 2023Stochastic interpolants with data-dependent couplingscited by 100
  16. 2024Generator Matching: Generative modeling with arbitrary Markov processescited by 86
  17. 2023Non-Denoising Forward-Time Diffusionscited by 77
  18. 2022From Denoising Diffusions to Denoising Markov Modelscited by 58
  19. 2024Flow Matching with General Discrete Paths: A Kinetic-Optimal Perspectivecited by 57
  20. 2025Edit Flows: Flow Matching with Edit Operationscited by 55
  21. 2025Unified Multimodal Discrete Diffusioncited by 49
  22. 2023Trans-Dimensional Generative Modeling via Jump Diffusion Modelscited by 35
  23. 2025Esoteric Language Models: A Family of Any-Order Diffusion LLMscited by 34
  24. 2025LEAPS: A discrete neural sampler via locally equivariant networkscited by 31
  25. 2025Learning-Order Autoregressive Models with Application to Molecular Graph Generationcited by 29
  26. 2023Transport meets Variational Inference: Controlled Monte Carlo Diffusionscited by 21
  27. 2023Multimarginal generative modeling with stochastic interpolantscited by 17

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 12; 2026: 39.

51 papers.

Sort:

By area:

  1. 2025Diffusion LLM with Native Variable Generation Lengths: Let [EOS] Lead the Waycited by 22
  2. 2025Attention Sinks in Diffusion Language Modelscited by 20
  3. 2026DreamOn: Diffusion Language Models For Code Infilling Beyond Fixed-size Canvascited by 19
  4. 2025OneFlow: Concurrent Mixed-Modal and Interleaved Generation with Edit Flowscited by 17
  5. 2025CreditDecoding: Accelerating Parallel Decoding in Diffusion Large Language Models with Trace Creditcited by 13
  6. 2025Rainbow Padding: Mitigating Early Termination in Instruction-Tuned Diffusion LLMscited by 10
  7. 2026CORE: Context-Robust Remasking for Diffusion Language Modelscited by 10
  8. 2026Stop Training for the Worst: Progressive Unmasking Accelerates Masked Diffusion Trainingcited by 10
  9. 2026LLaDA-o: An Effective and Length-Adaptive Omni Diffusion Modelcited by 10
  10. 2025Masks Can Be Distracting: On Context Comprehension in Diffusion Language Modelscited by 9
  11. 2026Where-to-Unmask: Ground-Truth-Guided Unmasking Order Learning for Masked Diffusion Language Modelscited by 9
  12. 2026ρ-textttEOS: Training-free Bidirectional Variable-Length Control for Masked Diffusion LLMscited by 7
38 more
  1. 2026Beyond Masks: Efficient, Flexible Diffusion Language Models via Deletion-Insertion Processescited by 7
  2. 2026When to Commit? Towards Variable-Size Self-Contained Blocks for Discrete Diffusion Language Modelscited by 7
  3. 2025Generative Modeling from Black-box Corruptions via Self-Consistent Stochastic Interpolantscited by 6
  4. 2025Finish First, Perfect Later: Test-Time Token-Level Cross-Validation for Diffusion Large Language Modelscited by 5
  5. 2026Seeing the End at Step Zero: Accelerating Diffusion MLLMs via MLP Sparsity-Aware Truncationcited by 5
  6. 2025Rethinking Reasoning with MDLMs: Early Exits, Post-hoc Reasoning, and Beyondcited by 4
  7. 2025XLM: A Python package for non-autoregressive language modelscited by 4
  8. 2026Improving Variable-Length Generation in Diffusion Language Models via Length Regularizationcited by 4
  9. 2025A Comprehensive Study on Visual Token Redundancy for Discrete Diffusion-based Multimodal Large Language Modelscited by 3
  10. 2026Swordsman: Entropy-Driven Adaptive Block Partition for Efficient Diffusion Language Modelscited by 3
  11. 2026Predict, Don't Iterate: Efficient Adaptive-Length Infilling for Diffusion Language Modelscited by 3
  12. 2025Foundations of Diffusion Models in General State Spaces: A Self-Contained Introductioncited by 2
  13. 2026Diffusion LMs Can Approximate Optimal Infilling Lengths Implicitlycited by 2
  14. 2026Discrete Tilt Matchingcited by 2
  15. 2026Edit-Based Refinement for Parallel Masked Diffusion Language Modelscited by 2
  16. 2026Generative Molecular Morphing for Flexible-Size Design via Unbalanced Optimal Transportcited by 2
  17. 2026A2D2: Fine-Tuning Any-Length Discrete Diffusion for Adaptive Decodingcited by 2
  18. 2026Adaptive Block Diffusion: Resolving Training-Inference Mismatch in Diffusion Language Modelscited by 2
  19. 2026Set Diffusion: Interpolating Token Orderings Between Autoregression and Diffusion for Fast and Flexible Decodingcited by 2
  20. 2026Insertion Based Sequence Generation with Learnable Order Dynamicscited by 1
  21. 2026Gumbel Distillation for Parallel Text Generationcited by 1
  22. 2026GeoBlock: Inferring Block Granularity from Dependency Geometry in Diffusion Language Modelscited by 1
  23. 2026Dystruct: Dynamically Structured Diffusion Language Model Decoding via Bayesian Inferencecited by 1
  24. 2026VoidPadding: Let [VOID] Handle Padding in Masked Diffusion Language Models so that [EOS] Can Focus on Semantic Terminationcited by 1
  25. 2026Multi-Mask Diffusion Language Models for Few-Step Generationcited by 1
  26. 2026Expanding Flow Mapscited by 1
  27. 2026From Interface to Inference: Eliciting Any-Order Inference from Any-Order Modelscited by 1
  28. 2026Reconsidering Positional Supervision in Masked Diffusion Language Model Trainingcited by 0
  29. 2026Understanding and Accelerating the Training of Masked Diffusion Language Modelscited by 0
  30. 2026Latent Process Generator Matchingcited by 0
  31. 2026What Gets Unmasked First? Trajectory Analysis of Diffusion Models for Graph-to-Text Generationcited by 0
  32. 2026BlockGen: Flexible Blockwise Sequence Modeling with Hybrid Samplerscited by 0
  33. 2026A Continuous-Time Markov Chain Framework for Insertion Language Modelscited by 0
  34. 2026Length-Adaptive Decoding for Masked Diffusion Machine Translationcited by 0
  35. 2026CARVE: Verified Expansion for Variable-Length Generation in Diffusion Language Modelscited by 0
  36. 2026How Should Diffusion Language Models Edit Code?cited by 0
  37. 2026Plan Canvas: Fixed Reasoning Regions for Continuous Language Flowscited by 0
  38. 2026FragFlex: Fragment-Based Flexible Molecular Generation for Exploring Vast Chemical Spacecited by 0

50 most cited of 51 All 51 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Recursive Scaling in Masked Diffusion Models
  2. 2025Error Bounds and Optimal Schedules for Masked Diffusions with Factorized Approximations
  3. 2025Beyond Masked and Unmasked: Discrete Diffusion Models via Partial Masking
  4. 2026DUEL: Exact Likelihood for Masked Diffusion via Deterministic Unmasking
  5. 2025Masked Diffusion Models are Secretly Learned-Order Autoregressive Models
  6. 2026Transformer-Based Masked Diffusion Models Provably Learn Distributions with Latent Structure
  7. 2025Any-Order GPT as Masked Diffusion Model: Decoupling Formulation and Architecture
  8. 2026Learning Generation Orders for Masked Discrete Diffusion Models via Variational Inference