mlsanityData to 7 Oct 2026

Discrete Flow Matching

arXiv 2407.15595 ↗First posted 22 Jul 2024cs.LG cs.AIcited by 301

On the map: Diffusion models → Flow matching, time series and science

Abstract (arXiv)

Despite Flow Matching and diffusion models having emerged as powerful generative paradigms for continuous variables such as images and videos, their application to high-dimensional discrete data, such as language, is still limited. In this work, we present Discrete Flow Matching, a novel discrete flow paradigm designed specifically for generating discrete data. Discrete Flow Matching offers several key contributions:(i) it works with a general family of probability paths interpolating between source and target distributions; (ii) it allows for a generic formula for sampling from these probability paths using learned posteriors such as the probability denoiser (x-prediction) and noise-prediction (ε-prediction); (iii) practically, focusing on specific probability paths defined with different schedulers improves generative perplexity compared to previous discrete diffusion and flow models; and (iv) by scaling Discrete Flow Matching models up to 1.7B parameters, we reach 6.7% Pass@1 and 13.4% Pass@10 on HumanEval and 6.7% Pass@1 and 20.6% Pass@10 on 1-shot MBPP coding benchmarks. Our approach is capable of generating high-quality discrete data in a non-autoregressive fashion, significantly closing the gap between autoregressive models and discrete flow models.

Key facts. Cited by 301 papers in the map (data through 7 Oct 2026). Placed in Flow matching, time series and science, one of 14 directions in Diffusion models. Builds on 44 of its 58 references that are in the map. First posted on arXiv 22 Jul 2024.

Builds on

44 of 58 references.

Closest ancestors · 6

Same direction, newest first

  1. 2024Dirichlet Flow Matching with Applications to DNA Sequence Design8 Feb
  2. 2024Generative Flows on Discrete State-Spaces: Enabling Multimodal Flows with Applications to Protein Co-Design7 Feb
  3. 2023Multisample Flow Matching: Straightening Flows with Minibatch Couplings28 Apr
  4. 2023Improving and generalizing flow-based generative models with minibatch optimal transport1 Feb
  5. 2022Flow Matching for Generative Modeling6 Oct
  6. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow7 Sep

Other references · 38

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  5. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  6. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  7. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  8. 2022Scalable Diffusion Models with Transformerscited by 5,931
  9. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  10. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
  11. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  12. 2021Program Synthesis with Large Language Modelscited by 3,509
26 more
  1. 2023A Survey of Large Language Modelscited by 2,537
  2. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,526
  3. 2023Code Llama: Open Foundation Models for Codecited by 2,105
  4. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,725
  5. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,665
  6. 2022MaskGIT: Masked Generative Image Transformercited by 1,082
  7. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,006
  8. 2022Building Normalizing Flows with Stochastic Interpolantscited by 964
  9. 2023StarCoder: may the source be with you!cited by 790
  10. 2023Muse: Text-To-Image Generation via Masked Generative Transformerscited by 640
  11. 2021Argmax Flows and Multinomial Diffusion: Learning Categorical Distributionscited by 566
  12. 2023Simple and Controllable Music Generationcited by 480
  13. 2022A Continuous Time Framework for Discrete Denoising Modelscited by 437
  14. 2022Analog Bits: Generating Discrete Data using Diffusion Models with Self-Conditioningcited by 362
  15. 2022AudioGen: Textually Guided Audio Generationcited by 292
  16. 2024Large Language Models for Mathematical Reasoning: Progresses and Challengescited by 283
  17. 2023MathPrompter: Mathematical Reasoning using Large Language Modelscited by 236
  18. 2022DiffusionBERT: Improving Generative Masked Language Models with Diffusion Modelscited by 214
  19. 2022Continuous diffusion for categorical datacited by 202
  20. 2022SSD-LM: Semi-autoregressive Simplex-based Diffusion Language Model for Text Generation and Modular Controlcited by 191
  21. 2022Latent Diffusion for Language Generationcited by 110
  22. 2021Step-unrolled Denoising Autoencoders for Text Generationcited by 102
  23. 2023Textually Pretrained Speech Language Modelscited by 99
  24. 2024Scientific Large Language Models: A Survey on Biological & Chemical Domainscited by 80
  25. 2024Masked Audio Generation using a Single Non-Autoregressive Transformercited by 50
  26. 2023A Pytorch Reproduction of Masked Generative Image Transformercited by 25

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 24; 2025: 134; 2026: 143.

301 papers.

Sort:

By area:

  1. 2025Large Language Diffusion Modelscited by 872
  2. 2024Transfusion: Predict the Next Token and Diffuse Images with One Multi-Modal Modelcited by 468
  3. 2025Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decodingcited by 404
  4. 2024Scaling up Masked Diffusion Models on Textcited by 195
  5. 2025Remasking Discrete Diffusion Models with Inference-Time Scalingcited by 181
  6. 2024Simple Guidance Mechanisms for Discrete Diffusion Modelscited by 148
  7. 2025LLaDA-V: Large Language Diffusion Models with Visual Instruction Tuningcited by 145
  8. 2025Accelerated Sampling from Masked Diffusion Models via Entropy Bounded Unmaskingcited by 128
  9. 2025Emu3.5: Native Multimodal Models are World Learnerscited by 119
  10. 2024PnP-Flow: Plug-and-Play Image Restoration with Flow Matchingcited by 92
  11. 2025Discrete Diffusion VLA: Bringing Discrete Diffusion to Action Decoding in Vision-Language-Action Policiescited by 90
  12. 2024Energy-Based Diffusion Language Models for Text Generationcited by 86
38 more
  1. 2025Unified Multimodal Understanding and Generation Models: Advances, Challenges, and Opportunitiescited by 81
  2. 2025Path Planning for Masked Diffusion Model Samplingcited by 79
  3. 2024Dual Diffusion for Unified Image Generation and Understandingcited by 76
  4. 2025A Survey on Diffusion Language Modelscited by 75
  5. 2024Think While You Generate: Discrete Diffusion with Planned Denoisingcited by 61
  6. 2025Discrete Diffusion in Large Language and Multimodal Models: A Surveycited by 61
  7. 2024Informed Correctors for Discrete Diffusion Modelscited by 57
  8. 2024Flow Matching with General Discrete Paths: A Kinetic-Optimal Perspectivecited by 54
  9. 2025Edit Flows: Flow Matching with Edit Operationscited by 53
  10. 2025Any-Order Flexible Length Masked Diffusioncited by 48
  11. 2025Fast Solvers for Discrete Diffusion Models: Theory and Applications of High-Order Algorithmscited by 47
  12. 2025Theoretical Benefit and Limitation of Diffusion Language Modelcited by 46
  13. 2025FUDOKI: Discrete Flow-based Unified Understanding and Generation via Kinetic-Optimal Velocitiescited by 46
  14. 2025On the Guidance of Flow Matchingcited by 44
  15. 2024textitJump Your Steps: Optimizing Sampling Schedule of Discrete Diffusion Modelscited by 40
  16. 2025Fine-Tuning Masked Diffusion for Provable Self-Correctioncited by 39
  17. 2026Categorical Flow Mapscited by 37
  18. 2024Gradient-Free Generation for Hard-Constrained Systemscited by 35
  19. 2024PepTune: De Novo Generation of Therapeutic Peptides with Multi-Objective-Guided Discrete Diffusioncited by 35
  20. 2025Continuously Augmented Discrete Diffusion model for Categorical Generative Modelingcited by 32
  21. 2025MoFlow: One-Step Flow Matching for Human Trajectory Forecasting via Implicit Maximum Likelihood Estimation based Distillationcited by 31
  22. 2025LEAPS: A discrete neural sampler via locally equivariant networkscited by 30
  23. 2025Improving Reasoning for Diffusion Language Models via Group Diffusion Policy Optimizationcited by 30
  24. 2025Continuous Diffusion Model for Language Modelingcited by 29
  25. 2024Steering Masked Discrete Diffusion Models via Discrete Denoising Posterior Predictioncited by 28
  26. 2025Coevolutionary Continuous Discrete Diffusion: Make Your Diffusion Language Model a Latent Reasonercited by 28
  27. 2025Open Materials Generation with Stochastic Interpolantscited by 26
  28. 2025Target Concrete Score Matching: A Holistic Framework for Discrete Diffusioncited by 25
  29. 2025Training-Free Guidance Beyond Differentiability: Scalable Path Steering with Tree Search in Diffusion and Flow Modelscited by 23
  30. 2025Set Block Decoding is a Language Model Inference Acceleratorcited by 22
  31. 2025MDNS: Masked Diffusion Neural Sampler via Stochastic Optimal Controlcited by 20
  32. 2025Discrete Diffusion Models: Novel Analysis and New Sampler Guaranteescited by 20
  33. 2025Beyond Masked and Unmasked: Discrete Diffusion Models via Partial Maskingcited by 19
  34. 2025Diffuse Everything: Multimodal Diffusion Models on Arbitrary State Spacescited by 19
  35. 2025FLOWR: Flow Matching for Structure-Aware De Novo, Interaction- and Fragment-Based Ligand Generationcited by 18
  36. 2025One-Step Flow Policy Mirror Descentcited by 18
  37. 2025α-Flow: A Unified Framework for Continuous-State Discrete Flow Matching Modelscited by 17
  38. 2025Anchored Diffusion Language Modelcited by 17

50 most cited of 301 All 301 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025A Theoretical Analysis of Discrete Flow Matching Generative Models
  2. 2025An Introduction to Flow Matching and Diffusion Models
  3. 2023Flow Matching in Latent Space
  4. 2025Contrastive Flow Matching
  5. 2025GLASS Flows: Transition Sampling for Alignment of Flow and Diffusion Models
  6. 2026Better Source, Better Flow: Learning Condition-Dependent Source Distribution for Flow Matching
  7. 2026Difficulty-Calibrated Interpolation Paths for Conditional Flow Matching
  8. 2025Flow Matching in the Low-Noise Regime: Pathologies and a Contrastive Remedy