mlsanityData to 7 Oct 2026

Dirichlet Diffusion Score Model for Biological Sequence Generation

arXiv 2305.10699 ↗First posted 18 May 2023cs.LG q-bio.GN q-bio.QMcited by 73

On the map: Diffusion models → Flow matching, time series and science

Abstract (arXiv)

Designing biological sequences is an important challenge that requires satisfying complex constraints and thus is a natural problem to address with deep generative modeling. Diffusion generative models have achieved considerable success in many applications. Score-based generative stochastic differential equations (SDE) model is a continuous-time diffusion model framework that enjoys many benefits, but the originally proposed SDEs are not naturally designed for modeling discrete data. To develop generative SDE models for discrete data such as biological sequences, here we introduce a diffusion process defined in the probability simplex space with stationary distribution being the Dirichlet distribution. This makes diffusion in continuous space natural for modeling discrete data. We refer to this approach as Dirchlet diffusion score model. We demonstrate that this technique can generate samples that satisfy hard constraints using a Sudoku generation task. This generative model can also solve Sudoku, including hard puzzles, without additional training. Finally, we applied this approach to develop the first human promoter DNA sequence design model and showed that designed sequences share similar properties with natural promoter sequences.

Key facts. Cited by 73 papers in the map (data through 7 Oct 2026). Placed in Flow matching, time series and science, one of 14 directions in Diffusion models. Builds on 22 of its 48 references that are in the map. First posted on arXiv 18 May 2023.

Builds on

22 of 48 references.

Closest ancestors · 2

Same direction, newest first

  1. 2022Diffusion probabilistic modeling of protein backbones in 3D for the motif-scaffolding problem8 Jun
  2. 2022Protein Structure and Sequence Generation with Equivariant Denoising Diffusion Probabilistic Models26 May

Other references · 20

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  3. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  4. 2018Neural Ordinary Differential Equationscited by 3,733
  5. 2020Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domainscited by 2,170
  6. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,665
  7. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,006
  8. 2021Maximum Likelihood Training of Score-Based Diffusion Modelscited by 670
  9. 2021Score-based Generative Modeling in Latent Spacecited by 634
  10. 2021Argmax Flows and Multinomial Diffusion: Learning Categorical Distributionscited by 566
  11. 2022A Continuous Time Framework for Discrete Denoising Modelscited by 437
  12. 2022Analog Bits: Generating Discrete Data using Diffusion Models with Self-Conditioningcited by 362
8 more
  1. 2021Autoregressive Diffusion Modelscited by 208
  2. 2021A Variational Perspective on Diffusion-Based Generative Models and Score Matchingcited by 182
  3. 2019SATNet: Bridging deep learning and logical reasoning using a differentiable satisfiability solvercited by 173
  4. 2022Score-based Continuous-time Discrete Diffusion Modelscited by 167
  5. 2022Latent Diffusion for Language Generationcited by 110
  6. 2017Recurrent Relational Networkscited by 71
  7. 2022Categorical SDEs with Simplex Diffusioncited by 42
  8. 2017Generating and designing DNA with deep generative modelscited by 30

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 4; 2024: 24; 2025: 23; 2026: 22.

73 papers.

Sort:

By area:

  1. 2024Simple and Effective Masked Diffusion Language Modelscited by 800
  2. 2024Simplified and Generalized Masked Diffusion for Discrete Datacited by 606
  3. 2025Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Modelscited by 385
  4. 2024Fine-Tuning of Continuous-Time Diffusion Models as Entropy-Regularized Controlcited by 136
  5. 2024Derivative-Free Guidance in Continuous and Discrete Diffusion Models with Soft Value-Based Decodingcited by 127
  6. 2024Unlocking Guidance for Discrete State-Space Diffusion and Flow Modelscited by 124
  7. 2024Dirichlet Flow Matching with Applications to DNA Sequence Designcited by 110
  8. 2024Understanding Reinforcement Learning-Based Fine-Tuning of Diffusion Models: A Tutorial and Reviewcited by 92
  9. 2024Fine-Tuning Discrete Diffusion Models via Reward Optimization with Applications to DNA and Protein Designcited by 72
  10. 2024Fisher Flow Matching for Generative Modeling over Discrete Datacited by 68
  11. 2023On the Design Fundamentals of Diffusion Models: A Surveycited by 59
  12. 2024Mixed Continuous and Categorical Flow Matching for 3D De Novo Molecule Generationcited by 53
38 more
  1. 2024Feedback Efficient Online Fine-Tuning of Diffusion Modelscited by 52
  2. 2024How Discrete and Continuous Diffusion Meet: Comprehensive Analysis of Discrete Diffusion Models via a Stochastic Integral Frameworkcited by 48
  3. 2025Fast Solvers for Discrete Diffusion Models: Theory and Applications of High-Order Algorithmscited by 47
  4. 2025Theoretical Benefit and Limitation of Diffusion Language Modelcited by 46
  5. 2024Categorical Flow Matching on Statistical Manifoldscited by 43
  6. 2024Accelerating Diffusion Models with Parallel Sampling: Inference at Sub-Linear Time Complexitycited by 42
  7. 2025Time Is a Feature: Exploiting Temporal Dynamics in Diffusion Language Modelscited by 33
  8. 2025Continuously Augmented Discrete Diffusion model for Categorical Generative Modelingcited by 32
  9. 2025Continuous Diffusion Model for Language Modelingcited by 29
  10. 2024Bridging Model-Based Optimization and Generative Modeling via Conservative Fine-Tuning of Diffusion Modelscited by 24
  11. 2025Training-Free Guidance Beyond Differentiability: Scalable Path Steering with Tree Search in Diffusion and Flow Modelscited by 23
  12. 2025GENERator: A Long-Context Generative Genomic Foundation Modelcited by 20
  13. 2023Beta Diffusioncited by 17
  14. 2025α-Flow: A Unified Framework for Continuous-State Discrete Flow Matching Modelscited by 17
  15. 2024Advancing Graph Generation through Beta Diffusioncited by 13
  16. 2025TR2-D2: Tree Search Guided Trajectory-Aware Fine-Tuning for Discrete Diffusioncited by 13
  17. 2026Continuous Diffusion Scales Competitively with Discrete Diffusion for Languagecited by 13
  18. 2023Generative Fractional Diffusion Modelscited by 12
  19. 2024DiscDiff: Latent Diffusion Model for DNA Sequence Generationcited by 12
  20. 2025Gumbel-Softmax Flow Matching with Straight-Through Guidance for Controllable Biological Sequence Generationcited by 11
  21. 2024HELM: Hierarchical Encoding for mRNA Language Modelingcited by 9
  22. 2023Latent Diffusion Model for DNA Sequence Generationcited by 8
  23. 2024Discrete Diffusion Language Model for Efficient Text Summarizationcited by 7
  24. 2024Generate and Instantiate What You Prefer: Text-Guided Diffusion for Sequential Recommendationcited by 6
  25. 2025Regulatory DNA sequence Design with Reinforcement Learningcited by 6
  26. 2026Kernel-Gradient Drifting Modelscited by 6
  27. 2026Hacking Generative Perplexity: Why Unconditional Text Evaluation Needs Distributional Metricscited by 6
  28. 2024Latent Diffusion Models for Controllable RNA Sequence Generationcited by 4
  29. 2025Inverse Flow and Consistency Modelscited by 4
  30. 2025DDPS: Discrete Diffusion Posterior Sampling for Paths in Layered Graphscited by 4
  31. 2025Evolvable Conditional Diffusioncited by 4
  32. 2025Discrete State Diffusion Models: A Sample Complexity Perspectivecited by 4
  33. 2026Adjoint Matching through the Lens of the Stochastic Maximum Principle in Optimal Controlcited by 4
  34. 2024scRDiT: Generating single-cell RNA-seq data by diffusion transformers and accelerating samplingcited by 3
  35. 2025Ctrl-DNA: Controllable Cell-Type-Specific Regulatory DNA Design via Constrained RLcited by 3
  36. 2025ShortListing Model: A Streamlined SimplexDiffusion for Discrete Variable Generationcited by 3
  37. 2025Simplex-to-Euclidean Bijections for Categorical Flow Matchingcited by 3
  38. 2025Fractional Diffusion Bridge Modelscited by 3

50 most cited of 73 All 73 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023Diffusion Models in Bioinformatics: A New Wave of Deep Learning Revolution in Action
  2. 2023Score-based Generative Modeling Through Backward Stochastic Differential Equations: Inversion and Generation
  3. 2024Combinatorial Complex Score-based Diffusion Modelling through Stochastic Differential Equations
  4. 2025Bit-Level Discrete Diffusion with Markov Probabilistic Models: An Improved Framework with Sharp Convergence Bounds under Minimal Assumptions
  5. 2022How Much is Enough? A Study on Diffusion Times in Score-based Generative Models
  6. 2026A Tutorial on Diffusion Theory: From Differential Equations to Diffusion Models
  7. 2022Score-based Generative Modeling of Graphs via the System of Stochastic Differential Equations
  8. 2024Generative Flows on Discrete State-Spaces: Enabling Multimodal Flows with Applications to Protein Co-Design