# Protein Discovery with Discrete Walk-Jump Sampling

arXiv: https://arxiv.org/abs/2306.12360 · HTML: https://mlsanity.com/paper/2306.12360/

## Key facts

- Cited by 38 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 55 references that are in the map.
- First posted on arXiv 8 Jun 2023.

## Builds on (most cited first, 17 of 17)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Structured Denoising Diffusion Models in Discrete State-Spaces](https://mlsanity.com/paper/2107.03006/) (2021)
- [Variational Diffusion Models](https://mlsanity.com/paper/2107.00630/) (2021)
- [E(n) Equivariant Graph Neural Networks](https://mlsanity.com/paper/2102.09844/) (2021)
- [Protein Design with Guided Discrete Diffusion](https://mlsanity.com/paper/2305.20009/) (2023)
- [Concrete Score Matching: Generalized Score Matching for Discrete Data](https://mlsanity.com/paper/2211.00802/) (2022)
- [Iterative Refinement Graph Neural Network for Antibody Sequence-Structure Co-design](https://mlsanity.com/paper/2110.04624/) (2021)
- [Conditional Antibody Design as 3D Equivariant Graph Translation](https://mlsanity.com/paper/2208.06073/) (2022)
- [Neural Machine Translation in Linear Time](https://mlsanity.com/paper/1610.10099/) (2016)
- [Neural Empirical Bayes](https://mlsanity.com/paper/1903.02334/) (2019)
- [Criteria of efficiency for conformal prediction](https://mlsanity.com/paper/1603.04416/) (2016)
- [Deep Energy Estimator Networks](https://mlsanity.com/paper/1805.08306/) (2018)
- [Multimeasurement Generative Models](https://mlsanity.com/paper/2112.09822/) (2021)
- [A Pareto-optimal compositional energy-based model for sampling and optimization of protein sequences](https://mlsanity.com/paper/2210.10838/) (2022)

## Built on by (most cited first, 30 of 38)

- [Unlocking Guidance for Discrete State-Space Diffusion and Flow Models](https://mlsanity.com/paper/2406.01572/) (2024)
- [Dirichlet Flow Matching with Applications to DNA Sequence Design](https://mlsanity.com/paper/2402.05841/) (2024)
- [Understanding Reinforcement Learning-Based Fine-Tuning of Diffusion Models: A Tutorial and Review](https://mlsanity.com/paper/2407.13734/) (2024)
- [Inference-Time Alignment in Diffusion Models with Reward-Guided Generation: Tutorial and Review](https://mlsanity.com/paper/2501.09685/) (2025)
- [How Discrete and Continuous Diffusion Meet: Comprehensive Analysis of Discrete Diffusion Models via a Stochastic Integral Framework](https://mlsanity.com/paper/2410.03601/) (2024)
- [Fast Solvers for Discrete Diffusion Models: Theory and Applications of High-Order Algorithms](https://mlsanity.com/paper/2502.00234/) (2025)
- [Sequence-Augmented SE(3)-Flow Matching For Conditional Protein Backbone Generation](https://mlsanity.com/paper/2405.20313/) (2024)
- [Full-Atom Peptide Design based on Multi-modal Flow Matching](https://mlsanity.com/paper/2406.00735/) (2024)
- [Improving Protein Optimization with Smoothed Fitness Landscapes](https://mlsanity.com/paper/2307.00494/) (2023)
- [Structure-based drug design by denoising voxel grids](https://mlsanity.com/paper/2405.03961/) (2024)
- [α-Flow: A Unified Framework for Continuous-State Discrete Flow Matching Models](https://mlsanity.com/paper/2504.10283/) (2025)
- [Concept Bottleneck Language Models For protein design](https://mlsanity.com/paper/2411.06090/) (2024)
- [Offline Model-Based Optimization: Comprehensive Review](https://mlsanity.com/paper/2503.17286/) (2025)
- [Robust Optimization in Protein Fitness Landscapes Using Reinforcement Learning in Latent Space](https://mlsanity.com/paper/2405.18986/) (2024)
- [UniMoMo: Unified Generative Modeling of 3D Molecules for De Novo Binder Design](https://mlsanity.com/paper/2503.19300/) (2025)
- [Diffusion on language model encodings for protein sequence generation](https://mlsanity.com/paper/2403.03726/) (2024)
- [Retrieval Augmented Diffusion Model for Structure-informed Antibody Design and Optimization](https://mlsanity.com/paper/2410.15040/) (2024)
- [Evolution Guided Generative Flow Networks](https://mlsanity.com/paper/2402.02186/) (2024)
- [Implicitly Guided Design with PropEn: Match your Data to Follow the Gradient](https://mlsanity.com/paper/2405.18075/) (2024)
- [JAMUN: Bridging Smoothed Molecular Dynamics and Score-Based Learning for Conformational Ensembles](https://mlsanity.com/paper/2410.14621/) (2024)
- [ProtFlow: Fast Protein Sequence Design via Flow Matching on Compressed Protein Language Model Embeddings](https://mlsanity.com/paper/2504.10983/) (2025)
- [DDPS: Discrete Diffusion Posterior Sampling for Paths in Layered Graphs](https://mlsanity.com/paper/2504.20754/) (2025)
- [Antibody DomainBed: Out-of-Distribution Generalization in Therapeutic Protein Design](https://mlsanity.com/paper/2407.21028/) (2024)
- [A noise-corrected Langevin algorithm and sampling by half-denoising](https://mlsanity.com/paper/2410.05837/) (2024)
- [Overconfident Oracles: Limitations of In Silico Sequence Design Benchmarking](https://mlsanity.com/paper/2502.17246/) (2025)
- [ProSpero: Active Learning for Robust Protein Design Beyond Wild-Type Neighborhoods](https://mlsanity.com/paper/2505.22494/) (2025)
- [Guided Sequence-Structure Generative Modeling for Iterative Antibody Optimization](https://mlsanity.com/paper/2509.16357/) (2025)
- [Reinforcement Learning for Sequence Design Leveraging Protein Language Models](https://mlsanity.com/paper/2407.03154/) (2024)
- [GROOT: Effective Design of Biological Sequences with Limited Experimental Data](https://mlsanity.com/paper/2411.11265/) (2024)
- [AffinityFlow: Guided Flows for Antibody Affinity Maturation](https://mlsanity.com/paper/2502.10365/) (2025)

## Abstract (arXiv)

We resolve difficulties in training and sampling from a discrete generative model by learning a smoothed energy function, sampling from the smoothed data manifold with Langevin Markov chain Monte Carlo (MCMC), and projecting back to the true data manifold with one-step denoising. Our Discrete Walk-Jump Sampling formalism combines the contrastive divergence training of an energy-based model and improved sample quality of a score-based model, while simplifying training and sampling by requiring only a single noise level. We evaluate the robustness of our approach on generative modeling of antibody proteins and introduce the distributional conformity score to benchmark protein generative models. By optimizing and sampling from our models for the proposed distributional conformity score, 97-100% of generated samples are successfully expressed and purified and 70% of functional designs show equal or improved binding affinity compared to known functional antibodies on the first attempt in a single round of laboratory experiments. We also report the first demonstration of long-run fast-mixing MCMC chains where diverse antibody protein classes are visited in a single MCMC chain.