# Gumbel-Softmax Flow Matching with Straight-Through Guidance for Controllable Biological Sequence Generation arXiv: https://arxiv.org/abs/2503.17361 ยท HTML: https://mlsanity.com/paper/2503.17361/ ## Key facts - Cited by 11 papers in the map (data through 14 Oct 2026). - Placed in Flow matching, time series and science, one of 14 directions in Diffusion models. - Builds on 31 of its 50 references that are in the map. - First posted on arXiv 21 Mar 2025. ## Placed in - Diffusion models / [Flow matching, time series and science](https://mlsanity.com/diffusion/flow-matching-time-series-and-science/) ## Builds on (most cited first, 30 of 31) - [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020) - [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022) - [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022) - [Flow Matching for Generative Modeling](https://mlsanity.com/paper/2210.02747/) (2022) - [RoFormer: Enhanced Transformer with Rotary Position Embedding](https://mlsanity.com/paper/2104.09864/) (2021) - [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019) - [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016) - [Optuna: A Next-generation Hyperparameter Optimization Framework](https://mlsanity.com/paper/1907.10902/) (2019) - [Structured Denoising Diffusion Models in Discrete State-Spaces](https://mlsanity.com/paper/2107.03006/) (2021) - [The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables](https://mlsanity.com/paper/1611.00712/) (2016) - [Stochastic Interpolants: A Unifying Framework for Flows and Diffusions](https://mlsanity.com/paper/2303.08797/) (2023) - [Improving and generalizing flow-based generative models with minibatch optimal transport](https://mlsanity.com/paper/2302.00482/) (2023) - [Simple and Effective Masked Diffusion Language Models](https://mlsanity.com/paper/2406.07524/) (2024) - [Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution](https://mlsanity.com/paper/2310.16834/) (2023) - [Simplified and Generalized Masked Diffusion for Discrete Data](https://mlsanity.com/paper/2406.04329/) (2024) - [A Continuous Time Framework for Discrete Denoising Models](https://mlsanity.com/paper/2205.14987/) (2022) - [Analog Bits: Generating Discrete Data using Diffusion Models with Self-Conditioning](https://mlsanity.com/paper/2208.04202/) (2022) - [Discrete Flow Matching](https://mlsanity.com/paper/2407.15595/) (2024) - [Generative Flows on Discrete State-Spaces: Enabling Multimodal Flows with Applications to Protein Co-Design](https://mlsanity.com/paper/2402.04997/) (2024) - [Rectified Flow: A Marginal Preserving Approach to Optimal Transport](https://mlsanity.com/paper/2209.14577/) (2022) - [Multisample Flow Matching: Straightening Flows with Minibatch Couplings](https://mlsanity.com/paper/2304.14772/) (2023) - [ProGen2: Exploring the Boundaries of Protein Language Models](https://mlsanity.com/paper/2206.13517/) (2022) - [Unlocking Guidance for Discrete State-Space Diffusion and Flow Models](https://mlsanity.com/paper/2406.01572/) (2024) - [Dirichlet Flow Matching with Applications to DNA Sequence Design](https://mlsanity.com/paper/2402.05841/) (2024) - [Non-Denoising Forward-Time Diffusions](https://mlsanity.com/paper/2312.14589/) (2023) - [Dirichlet Diffusion Score Model for Biological Sequence Generation](https://mlsanity.com/paper/2305.10699/) (2023) - [Fisher Flow Matching for Generative Modeling over Discrete Data](https://mlsanity.com/paper/2405.14664/) (2024) - [PepTune: De Novo Generation of Therapeutic Peptides with Multi-Objective-Guided Discrete Diffusion](https://mlsanity.com/paper/2412.17780/) (2024) - [Trajectory Flow Matching with Applications to Clinical Time Series Modeling](https://mlsanity.com/paper/2410.21154/) (2024) - [Learning to design protein-protein interactions with enhanced generalization](https://mlsanity.com/paper/2310.18515/) (2023) ## Built on by (most cited first, 11 of 11) - [Multi-Objective-Guided Discrete Flow Matching for Controllable Biological Sequence Design](https://mlsanity.com/paper/2505.07086/) (2025) - [Flow Matching Meets Biology and Life Science: A Survey](https://mlsanity.com/paper/2507.17731/) (2025) - [Steering Generative Models with Experimental Data for Protein Fitness Optimization](https://mlsanity.com/paper/2505.15093/) (2025) - [Soft-Di[M]O: Improving One-Step Discrete Image Generation with Soft Embeddings](https://mlsanity.com/paper/2509.22925/) (2025) - [AReUReDi: Annealed Rectified Updates for Refining Discrete Flows with Multi-Objective Guidance](https://mlsanity.com/paper/2510.00352/) (2025) - [Simplex-to-Euclidean Bijections for Categorical Flow Matching](https://mlsanity.com/paper/2510.27480/) (2025) - [Discrete Beckmann Transport Models for One-Step Language Modeling and Reasoning](https://mlsanity.com/paper/2609.15903/) (2026) - [A Unification of Discrete, Gaussian, and Simplicial Diffusion](https://mlsanity.com/paper/2512.15923/) (2025) - [Expanding Flow Maps](https://mlsanity.com/paper/2607.21585/) (2026) - [Generative Modeling of Discrete Data Using Geometric Latent Subspaces](https://mlsanity.com/paper/2601.21831/) (2026) - [Flexible Flows for Biological Sequence Design](https://mlsanity.com/paper/2606.10543/) (2026) ## Abstract (arXiv) Flow matching in the continuous simplex has emerged as a promising strategy for DNA sequence design, but struggles to scale to higher simplex dimensions required for peptide and protein generation. We introduce Gumbel-Softmax Flow and Score Matching, a generative framework on the simplex based on a novel Gumbel-Softmax interpolant with a time-dependent temperature. Using this interpolant, we introduce Gumbel-Softmax Flow Matching by deriving a parameterized velocity field that transports from smooth categorical distributions to distributions concentrated at a single vertex of the simplex. We alternatively present Gumbel-Softmax Score Matching which learns to regress the gradient of the probability density. Our framework enables high-quality, diverse generation and scales efficiently to higher-dimensional simplices. To enable training-free guidance, we propose Straight-Through Guided Flows (STGFlow), a classifier-based guidance method that leverages straight-through estimators to steer the unconditional velocity field toward optimal vertices of the simplex. STGFlow enables efficient inference-time guidance using classifiers pre-trained on clean sequences, and can be used with any discrete flow method. Together, these components form a robust framework for controllable de novo sequence generation. We demonstrate state-of-the-art performance in conditional DNA promoter design, sequence-only protein generation, and target-binding peptide design for rare disease treatment.