# Any-Order Flexible Length Masked Diffusion

arXiv: https://arxiv.org/abs/2509.01025 · HTML: https://mlsanity.com/paper/2509.01025/

## Key facts

- Cited by 52 papers in the map (data through 14 Oct 2026).
- Placed in Discrete and text diffusion, one of 14 directions in Diffusion models.
- Builds on 56 of its 60 references that are in the map.
- First posted on arXiv 31 Aug 2025.

## Placed in

- Diffusion models / [Discrete and text diffusion](https://mlsanity.com/diffusion/discrete-and-text-diffusion/)

## Builds on (most cited first, 30 of 56)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow](https://mlsanity.com/paper/2209.03003/) (2022)
- [FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness](https://mlsanity.com/paper/2205.14135/) (2022)
- [Qwen2 Technical Report](https://mlsanity.com/paper/2407.10671/) (2024)
- [Structured Denoising Diffusion Models in Discrete State-Spaces](https://mlsanity.com/paper/2107.03006/) (2021)
- [MaskGIT: Masked Generative Image Transformer](https://mlsanity.com/paper/2202.04200/) (2022)
- [Planning with Diffusion for Flexible Behavior Synthesis](https://mlsanity.com/paper/2205.09991/) (2022)
- [Building Normalizing Flows with Stochastic Interpolants](https://mlsanity.com/paper/2209.15571/) (2022)
- [Stochastic Interpolants: A Unifying Framework for Flows and Diffusions](https://mlsanity.com/paper/2303.08797/) (2023)
- [Large Language Diffusion Models](https://mlsanity.com/paper/2502.09992/) (2025)
- [Simple and Effective Masked Diffusion Language Models](https://mlsanity.com/paper/2406.07524/) (2024)
- [Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution](https://mlsanity.com/paper/2310.16834/) (2023)
- [DeepSeek LLM: Scaling Open-Source Language Models with Longtermism](https://mlsanity.com/paper/2401.02954/) (2024)
- [Simplified and Generalized Masked Diffusion for Discrete Data](https://mlsanity.com/paper/2406.04329/) (2024)
- [Argmax Flows and Multinomial Diffusion: Learning Categorical Distributions](https://mlsanity.com/paper/2102.05379/) (2021)
- [A Continuous Time Framework for Discrete Denoising Models](https://mlsanity.com/paper/2205.14987/) (2022)
- [Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decoding](https://mlsanity.com/paper/2505.22618/) (2025)
- [Your Absorbing Discrete Diffusion Secretly Models the Conditional Distributions of Clean Data](https://mlsanity.com/paper/2406.03736/) (2024)
- [Discrete Flow Matching](https://mlsanity.com/paper/2407.15595/) (2024)
- [Generative Flows on Discrete State-Spaces: Enabling Multimodal Flows with Applications to Protein Co-Design](https://mlsanity.com/paper/2402.04997/) (2024)
- [Scaling Diffusion Language Models via Adaptation from Autoregressive Models](https://mlsanity.com/paper/2410.17891/) (2024)
- [Efficient Training of Language Models to Fill in the Middle](https://mlsanity.com/paper/2207.14255/) (2022)
- [Mercury: Ultra-Fast Language Models Based on Diffusion](https://mlsanity.com/paper/2506.17298/) (2025)
- [Train for the Worst, Plan for the Best: Understanding Token Ordering in Masked Diffusions](https://mlsanity.com/paper/2502.06768/) (2025)
- [Masked Diffusion Models are Secretly Time-Agnostic Masked Models and Exploit Inaccurate Categorical Sampling](https://mlsanity.com/paper/2409.02908/) (2024)

## Built on by (most cited first, 30 of 52)

- [Diffusion LLM with Native Variable Generation Lengths: Let [EOS] Lead the Way](https://mlsanity.com/paper/2510.24605/) (2025)
- [Attention Sinks in Diffusion Language Models](https://mlsanity.com/paper/2510.15731/) (2025)
- [DreamOn: Diffusion Language Models For Code Infilling Beyond Fixed-size Canvas](https://mlsanity.com/paper/2602.01326/) (2026)
- [OneFlow: Concurrent Mixed-Modal and Interleaved Generation with Edit Flows](https://mlsanity.com/paper/2510.03506/) (2025)
- [CreditDecoding: Accelerating Parallel Decoding in Diffusion Large Language Models with Trace Credit](https://mlsanity.com/paper/2510.06133/) (2025)
- [LLaDA-o: An Effective and Length-Adaptive Omni Diffusion Model](https://mlsanity.com/paper/2603.01068/) (2026)
- [Rainbow Padding: Mitigating Early Termination in Instruction-Tuned Diffusion LLMs](https://mlsanity.com/paper/2510.03680/) (2025)
- [CORE: Context-Robust Remasking for Diffusion Language Models](https://mlsanity.com/paper/2602.04096/) (2026)
- [Where-to-Unmask: Ground-Truth-Guided Unmasking Order Learning for Masked Diffusion Language Models](https://mlsanity.com/paper/2602.09501/) (2026)
- [Stop Training for the Worst: Progressive Unmasking Accelerates Masked Diffusion Training](https://mlsanity.com/paper/2602.10314/) (2026)
- [Masks Can Be Distracting: On Context Comprehension in Diffusion Language Models](https://mlsanity.com/paper/2511.21338/) (2025)
- [Beyond Masks: Efficient, Flexible Diffusion Language Models via Deletion-Insertion Processes](https://mlsanity.com/paper/2603.23507/) (2026)
- [ρ-textttEOS: Training-free Bidirectional Variable-Length Control for Masked Diffusion LLMs](https://mlsanity.com/paper/2601.22527/) (2026)
- [When to Commit? Towards Variable-Size Self-Contained Blocks for Discrete Diffusion Language Models](https://mlsanity.com/paper/2604.23994/) (2026)
- [On Powerful Ways to Generate: Autoregression, Diffusion, and Beyond](https://mlsanity.com/paper/2510.06190/) (2025)
- [Generative Modeling from Black-box Corruptions via Self-Consistent Stochastic Interpolants](https://mlsanity.com/paper/2512.10857/) (2025)
- [Finish First, Perfect Later: Test-Time Token-Level Cross-Validation for Diffusion Large Language Models](https://mlsanity.com/paper/2510.05090/) (2025)
- [Seeing the End at Step Zero: Accelerating Diffusion MLLMs via MLP Sparsity-Aware Truncation](https://mlsanity.com/paper/2607.14557/) (2026)
- [Rethinking Reasoning with MDLMs: Early Exits, Post-hoc Reasoning, and Beyond](https://mlsanity.com/paper/2510.19990/) (2025)
- [XLM: A Python package for non-autoregressive language models](https://mlsanity.com/paper/2512.17065/) (2025)
- [Improving Variable-Length Generation in Diffusion Language Models via Length Regularization](https://mlsanity.com/paper/2602.07546/) (2026)
- [Predict, Don't Iterate: Efficient Adaptive-Length Infilling for Diffusion Language Models](https://mlsanity.com/paper/2609.02108/) (2026)
- [A Comprehensive Study on Visual Token Redundancy for Discrete Diffusion-based Multimodal Large Language Models](https://mlsanity.com/paper/2511.15098/) (2025)
- [Swordsman: Entropy-Driven Adaptive Block Partition for Efficient Diffusion Language Models](https://mlsanity.com/paper/2602.04399/) (2026)
- [Foundations of Diffusion Models in General State Spaces: A Self-Contained Introduction](https://mlsanity.com/paper/2512.05092/) (2025)
- [Diffusion LMs Can Approximate Optimal Infilling Lengths Implicitly](https://mlsanity.com/paper/2602.00476/) (2026)
- [Discrete Tilt Matching](https://mlsanity.com/paper/2604.18739/) (2026)
- [Edit-Based Refinement for Parallel Masked Diffusion Language Models](https://mlsanity.com/paper/2605.09603/) (2026)
- [Generative Molecular Morphing for Flexible-Size Design via Unbalanced Optimal Transport](https://mlsanity.com/paper/2606.07239/) (2026)
- [A2D2: Fine-Tuning Any-Length Discrete Diffusion for Adaptive Decoding](https://mlsanity.com/paper/2606.13565/) (2026)

## Abstract (arXiv)

Masked diffusion models (MDMs) have recently emerged as a promising alternative to autoregressive models over discrete domains. MDMs generate sequences in an any-order, parallel fashion, enabling fast inference and strong performance on non-causal tasks. However, a crucial limitation is that they do not support token insertions and are thus limited to fixed-length generations. To this end, we introduce Flexible Masked Diffusion Models (FlexMDMs), a discrete diffusion paradigm that simultaneously can model sequences of flexible length while provably retaining MDMs' flexibility of any-order inference. Grounded in an extension of the stochastic interpolant framework, FlexMDMs generate sequences by inserting mask tokens and unmasking them. Empirically, we show that FlexMDMs match MDMs in perplexity while modeling length statistics with much higher fidelity. On a synthetic maze planning task, they achieve ≈ 60 % higher success rate than MDM baselines. Finally, we show pretrained MDMs can easily be retrofitted into FlexMDMs: on 16 H100s, it takes only three days to fine-tune LLaDA-8B into a FlexMDM, achieving superior performance on math (GSM8K, 58% → 67%) and code infilling performance (52% → 65%).