# Importance Weighted Expectation-Maximization for Protein Sequence Design

arXiv: https://arxiv.org/abs/2305.00386 · HTML: https://mlsanity.com/paper/2305.00386/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 77 references that are in the map.
- First posted on arXiv 30 Apr 2023.

## Builds on (most cited first, 17 of 17)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Evaluating Protein Transfer Learning with TAPE](https://mlsanity.com/paper/1906.08230/) (2019)
- [Biological Sequence Design with GFlowNets](https://mlsanity.com/paper/2203.04115/) (2022)
- [ProGen: Language Modeling for Protein Generation](https://mlsanity.com/paper/2004.03497/) (2020)
- [Optimus: Organizing Sentences via Pre-trained Modeling of a Latent Space](https://mlsanity.com/paper/2004.04092/) (2020)
- [Conditioning by adaptive sampling for robust design](https://mlsanity.com/paper/1901.10060/) (2019)
- [Glancing Transformer for Non-Autoregressive Neural Machine Translation](https://mlsanity.com/paper/2008.07905/) (2020)
- [Design-Bench: Benchmarks for Data-Driven Offline Model-Based Optimization](https://mlsanity.com/paper/2202.08450/) (2022)
- [Iterative Amortized Inference](https://mlsanity.com/paper/1807.09356/) (2018)
- [Model Inversion Networks for Model-Based Optimization](https://mlsanity.com/paper/1912.13464/) (2019)
- [AdaLead: A simple and robust adaptive greedy search algorithm for sequence design](https://mlsanity.com/paper/2010.02141/) (2020)
- [Optimizing Molecules using Efficient Queries from Property Evaluations](https://mlsanity.com/paper/2011.01921/) (2020)
- [Design by adaptive sampling](https://mlsanity.com/paper/1810.03714/) (2018)
- [Unifying Likelihood-free Inference with Black-box Optimization and Beyond](https://mlsanity.com/paper/2110.03372/) (2021)
- [Reweighted Expectation Maximization](https://mlsanity.com/paper/1906.05850/) (2019)
- [Benchmarking deep generative models for diverse antibody sequence design](https://mlsanity.com/paper/2111.06801/) (2021)

## Built on by (most cited first, 12 of 12)

- [Improving Protein Optimization with Smoothed Fitness Landscapes](https://mlsanity.com/paper/2307.00494/) (2023)
- [SurfPro: Functional Protein Design Based on Continuous Surface](https://mlsanity.com/paper/2405.06693/) (2024)
- [Improved Off-policy Reinforcement Learning in Biological Sequence Design](https://mlsanity.com/paper/2410.04461/) (2024)
- [Steering Generative Models with Experimental Data for Protein Fitness Optimization](https://mlsanity.com/paper/2505.15093/) (2025)
- [Generative Enzyme Design Guided by Functionally Important Sites and Small-Molecule Substrates](https://mlsanity.com/paper/2405.08205/) (2024)
- [Cell Morphology-Guided Small Molecule Generation with GFlowNets](https://mlsanity.com/paper/2408.05196/) (2024)
- [Joint Design of Protein Sequence and Structure based on Motifs](https://mlsanity.com/paper/2310.02546/) (2023)
- [Overconfident Oracles: Limitations of In Silico Sequence Design Benchmarking](https://mlsanity.com/paper/2502.17246/) (2025)
- [ProSpero: Active Learning for Robust Protein Design Beyond Wild-Type Neighborhoods](https://mlsanity.com/paper/2505.22494/) (2025)
- [Functional Geometry Guided Protein Sequence and Backbone Structure Co-Design](https://mlsanity.com/paper/2310.04343/) (2023)
- [TaxDiff: Taxonomic-Guided Diffusion Model for Protein Sequence Generation](https://mlsanity.com/paper/2402.17156/) (2024)
- [Efficient Protein Optimization via Structure-aware Hamiltonian Dynamics](https://mlsanity.com/paper/2601.11012/) (2026)

## Abstract (arXiv)

Designing protein sequences with desired biological function is crucial in biology and chemistry. Recent machine learning methods use a surrogate sequence-function model to replace the expensive wet-lab validation. How can we efficiently generate diverse and novel protein sequences with high fitness? In this paper, we propose IsEM-Pro, an approach to generate protein sequences towards a given fitness criterion. At its core, IsEM-Pro is a latent generative model, augmented by combinatorial structure features from a separately learned Markov random fields (MRFs). We develop an Monte Carlo Expectation-Maximization method (MCEM) to learn the model. During inference, sampling from its latent space enhances diversity while its MRFs features guide the exploration in high fitness regions. Experiments on eight protein sequence design tasks show that our IsEM-Pro outperforms the previous best methods by at least 55% on average fitness score and generates more diverse and novel protein sequences.