mlsanityData to 7 Oct 2026

Generative Flows on Discrete State-Spaces: Enabling Multimodal Flows with Applications to Protein Co-Design

arXiv 2402.04997 ↗First posted 7 Feb 2024stat.ML cs.LG q-bio.QMcited by 294

On the map: Diffusion models → Flow matching, time series and science

Abstract (arXiv)

Combining discrete and continuous data is an important capability for generative models. We present Discrete Flow Models (DFMs), a new flow-based model of discrete data that provides the missing link in enabling flow-based generative models to be applied to multimodal continuous and discrete data problems. Our key insight is that the discrete equivalent of continuous space flow matching can be realized using Continuous Time Markov Chains. DFMs benefit from a simple derivation that includes discrete diffusion models as a specific instance while allowing improved performance over existing diffusion-based approaches. We utilize our DFMs method to build a multimodal flow-based modeling framework. We apply this capability to the task of protein co-design, wherein we learn a model for jointly generating protein structure and sequence. Our approach achieves state-of-the-art co-design performance while allowing the same multimodal model to be used for flexible generation of the sequence or structure.

Key facts. Cited by 294 papers in the map (data through 7 Oct 2026). Placed in Flow matching, time series and science, one of 14 directions in Diffusion models. Builds on 54 of its 84 references that are in the map. First posted on arXiv 7 Feb 2024.

Builds on

54 of 84 references.

Closest ancestors · 14

Same direction, newest first

  1. 2023Fast non-autoregressive inverse folding with discrete diffusion5 Dec
  2. 2023Sparse Training of Discrete Diffusion Models for Graph Generation3 Nov
  3. 2023SE(3)-Stochastic Flow Matching for Protein Backbone Generation3 Oct
  4. 2023Graph Denoising Diffusion for Inverse Protein Folding29 Jun
  5. 2023Protein Design with Guided Discrete Diffusion31 May
  6. 2023MiDi: Mixed Graph and 3D Denoising Diffusion for Molecule Generation17 Feb
  7. 2023Flow Matching on General Geometries7 Feb
  8. 2023SE(3) diffusion model with application to protein backbone generation5 Feb
  9. 2022Flow Matching for Generative Modeling6 Oct
  10. 2022TabDDPM: Modelling Tabular Data with Diffusion Models30 Sep
  11. 2022DiGress: Discrete Denoising diffusion for graph generation29 Sep
  12. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow7 Sep
2 more
  1. 2022Diffusion probabilistic modeling of protein backbones in 3D for the motif-scaffolding problem8 Jun
  2. 2022Protein Structure and Sequence Generation with Equivariant Denoising Diffusion Probabilistic Models26 May

Other references · 40

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  4. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  5. 2013Auto-Encoding Variational Bayescited by 3,980
  6. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 2,994
  7. 2017Decoupled Weight Decay Regularizationcited by 1,722
  8. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,665
  9. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,006
  10. 2022Building Normalizing Flows with Stochastic Interpolantscited by 964
  11. 2023Discrete Diffusion Modeling by Estimating the Ratios of the Data Distributioncited by 712
  12. 2024SiT: Exploring Flow and Diffusion-based Generative Models with Scalable Interpolant Transformerscited by 707
28 more
  1. 2023Scaling Vision Transformers to 22 Billion Parameterscited by 690
  2. 2021Argmax Flows and Multinomial Diffusion: Learning Categorical Distributionscited by 566
  3. 2023Reinforced Self-Training (ReST) for Language Modelingcited by 486
  4. 2022DiffuSeq: Sequence to Sequence Text Generation with Diffusion Modelscited by 439
  5. 2022A Continuous Time Framework for Discrete Denoising Modelscited by 437
  6. 2022Analog Bits: Generating Discrete Data using Diffusion Models with Self-Conditioningcited by 362
  7. 2021GFlowNet Foundationscited by 286
  8. 2022Continuous diffusion for categorical datacited by 202
  9. 2022SSD-LM: Semi-autoregressive Simplex-based Diffusion Language Model for Text Generation and Modular Controlcited by 191
  10. 2021A Variational Perspective on Diffusion-Based Generative Models and Score Matchingcited by 182
  11. 2022Score-based Continuous-time Discrete Diffusion Modelscited by 167
  12. 2023Likelihood-Based Diffusion Language Modelscited by 147
  13. 2022Concrete Score Matching: Generalized Score Matching for Discrete Datacited by 144
  14. 2022Self-conditioned Embedding Diffusion for Text Generationcited by 114
  15. 2023Restart Sampling for Improving Generative Processescited by 81
  16. 2023MolDiff: Addressing the Atom-Bond Inconsistency Problem in 3D Molecule Diffusion Generationcited by 65
  17. 2022Improved Vector Quantized Diffusion Modelscited by 62
  18. 2023Generating Novel, Designable, and Diverse Protein Structures by Equivariantly Diffusing Oriented Residue Cloudscited by 53
  19. 2022Categorical SDEs with Simplex Diffusioncited by 42
  20. 2023On Kinetic Optimal Probability Paths for Generative Modelscited by 42
  21. 2023MUDiff: Unified Diffusion for Complete Molecule Generationcited by 41
  22. 2023RetroBridge: Modeling Retrosynthesis with Markov Bridgescited by 35
  23. 2022Protein Sequence and Structure Co-Design with Equivariant Translationcited by 31
  24. 2023Blackout Diffusion: Generative Diffusion Models in Discrete-State Spacescited by 29
  25. 2023Multimarginal generative modeling with stochastic interpolantscited by 17
  26. 2023Exploring the Optimal Choice for Generative Processes in Diffusion Models: Ordinary vs Stochastic Differential Equationscited by 15
  27. 2023Diffusion on the Probability Simplexcited by 14
  28. 2023Formulating Discrete Probability Flow Through Optimal Transportcited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 54; 2025: 127; 2026: 113.

294 papers.

Sort:

By area:

  1. 2024Simple and Effective Masked Diffusion Language Modelscited by 800
  2. 2024Simplified and Generalized Masked Diffusion for Discrete Datacited by 606
  3. 2024Discrete Flow Matchingcited by 301
  4. 2025Remasking Discrete Diffusion Models with Inference-Time Scalingcited by 181
  5. 2024Simple Guidance Mechanisms for Discrete Diffusion Modelscited by 148
  6. 2025LLaDA-V: Large Language Diffusion Models with Visual Instruction Tuningcited by 145
  7. 2025Accelerated Sampling from Masked Diffusion Models via Entropy Bounded Unmaskingcited by 128
  8. 2024Derivative-Free Guidance in Continuous and Discrete Diffusion Models with Soft Value-Based Decodingcited by 127
  9. 2024Unlocking Guidance for Discrete State-Space Diffusion and Flow Modelscited by 124
  10. 2024Consistency Flow Matching: Defining Straight Flows with Velocity Consistencycited by 101
  11. 2025Dimple: Discrete Diffusion Multimodal Large Language Model with Parallel Decodingcited by 101
  12. 2024Understanding Reinforcement Learning-Based Fine-Tuning of Diffusion Models: A Tutorial and Reviewcited by 92
38 more
  1. 2024Energy-Based Diffusion Language Models for Text Generationcited by 86
  2. 2025Inference-Time Alignment in Diffusion Models with Reward-Guided Generation: Tutorial and Reviewcited by 82
  3. 2025Path Planning for Masked Diffusion Model Samplingcited by 79
  4. 2024Variational Flow Matching for Graph Generationcited by 78
  5. 2025All-atom Diffusion Transformers: Unified generative modelling of molecules and materialscited by 75
  6. 2024Fine-Tuning Discrete Diffusion Models via Reward Optimization with Applications to DNA and Protein Designcited by 72
  7. 2024Fisher Flow Matching for Generative Modeling over Discrete Datacited by 68
  8. 2025Proteina: Scaling Flow-based Protein Structure Generative Modelscited by 63
  9. 2024DPLM-2: A Multimodal Diffusion Protein Language Modelcited by 59
  10. 2024Flow Matching with General Discrete Paths: A Kinetic-Optimal Perspectivecited by 54
  11. 2024Mixed Continuous and Categorical Flow Matching for 3D De Novo Molecule Generationcited by 53
  12. 2025Edit Flows: Flow Matching with Edit Operationscited by 53
  13. 2024Generative Modeling of Molecular Dynamics Trajectoriescited by 48
  14. 2024How Discrete and Continuous Diffusion Meet: Comprehensive Analysis of Discrete Diffusion Models via a Stochastic Integral Frameworkcited by 48
  15. 2025Any-Order Flexible Length Masked Diffusioncited by 48
  16. 2025Fast Solvers for Discrete Diffusion Models: Theory and Applications of High-Order Algorithmscited by 47
  17. 2025Theoretical Benefit and Limitation of Diffusion Language Modelcited by 46
  18. 2025FUDOKI: Discrete Flow-based Unified Understanding and Generation via Kinetic-Optimal Velocitiescited by 46
  19. 2024Categorical Flow Matching on Statistical Manifoldscited by 43
  20. 2024Accelerating Diffusion Models with Parallel Sampling: Inference at Sub-Linear Time Complexitycited by 42
  21. 2024Meta Flow Matching: Integrating Vector Fields on the Wasserstein Manifoldcited by 41
  22. 2024SemlaFlow -- Efficient 3D Molecular Generation with Latent Attention and Equivariant Flow Matchingcited by 40
  23. 2024textitJump Your Steps: Optimizing Sampling Schedule of Discrete Diffusion Modelscited by 40
  24. 2025Fine-Tuning Masked Diffusion for Provable Self-Correctioncited by 39
  25. 2025SPG: Sandwiched Policy Gradient for Masked Diffusion Language Modelscited by 38
  26. 2026Categorical Flow Mapscited by 37
  27. 2024Sequence-Augmented SE(3)-Flow Matching For Conditional Protein Backbone Generationcited by 36
  28. 2024The Superposition of Diffusion Models Using the Ito Density Estimatorcited by 36
  29. 2024Out of Many, One: Designing and Scaffolding Proteins at the Scale of the Structural Universe with Genie 2cited by 33
  30. 2025An Introduction to Flow Matching and Diffusion Modelscited by 33
  31. 2025LEAPS: A discrete neural sampler via locally equivariant networkscited by 30
  32. 2025Debiasing Guidance for Discrete Diffusion with Sequential Monte Carlocited by 29
  33. 2024Steering Masked Discrete Diffusion Models via Discrete Denoising Posterior Predictioncited by 28
  34. 2025Coevolutionary Continuous Discrete Diffusion: Make Your Diffusion Language Model a Latent Reasonercited by 28
  35. 2025Open Materials Generation with Stochastic Interpolantscited by 26
  36. 2025La-Proteina: Atomistic Protein Generation via Partially Latent Flow Matchingcited by 26
  37. 2025Target Concrete Score Matching: A Holistic Framework for Discrete Diffusioncited by 25
  38. 2024Bridging Model-Based Optimization and Generative Modeling via Conservative Fine-Tuning of Diffusion Modelscited by 24

50 most cited of 294 All 294 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Branching Flows: Discrete, Continuous, and Manifold Flow Matching with Splits and Deletions
  2. 2026Context-weighted Discrete Flow Matching
  3. 2025Multi-Objective-Guided Discrete Flow Matching for Controllable Biological Sequence Design
  4. 2025MarS-FM: Generative Modeling of Molecular Dynamics via Markov State Models
  5. 2025Flows, straight but not so fast: Exploring the design space of Rectified Flows in Protein Design
  6. 2025Demystify Protein Generation with Hierarchical Conditional Diffusion Models
  7. 2025From thermodynamics to protein design: Diffusion models for biomolecule generation towards autonomous protein engineering
  8. 2026Minimal-Action Discrete Schr"odinger Bridge Matching for Peptide Sequence Design