# Improving Inverse Folding for Peptide Design with Diversity-regularized Direct Preference Optimization

arXiv: https://arxiv.org/abs/2410.19471 · HTML: https://mlsanity.com/paper/2410.19471/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 40 references that are in the map.
- First posted on arXiv 25 Oct 2024.

## Builds on (most cited first, 13 of 13)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [Learning from Protein Structure with Geometric Vector Perceptrons](https://mlsanity.com/paper/2009.01411/) (2020)
- [Disentangling Length from Quality in Direct Preference Optimization](https://mlsanity.com/paper/2403.19159/) (2024)
- [Beyond Reverse KL: Generalizing Direct Preference Optimization with Diverse Divergence Constraints](https://mlsanity.com/paper/2309.16240/) (2023)
- [Beyond One-Preference-Fits-All Alignment: Multi-Objective Direct Preference Optimization](https://mlsanity.com/paper/2310.03708/) (2023)
- [PiFold: Toward effective and efficient protein inverse folding](https://mlsanity.com/paper/2209.12643/) (2022)
- [Equivariant Graph Neural Networks for 3D Macromolecular Structure](https://mlsanity.com/paper/2106.03843/) (2021)
- [Graph Denoising Diffusion for Inverse Protein Folding](https://mlsanity.com/paper/2306.16819/) (2023)
- [Knowledge-Design: Pushing the Limit of Protein Design via Knowledge Refinement](https://mlsanity.com/paper/2305.15151/) (2023)
- [Preference Optimization for Molecular Language Models](https://mlsanity.com/paper/2310.12304/) (2023)
- [Preference optimization of protein language models as a multi-objective binder design paradigm](https://mlsanity.com/paper/2403.04187/) (2024)

## Built on by (most cited first, 5 of 5)

- [Diverse Preference Optimization](https://mlsanity.com/paper/2501.18101/) (2025)
- [Improving Protein Sequence Design through Designability Preference Optimization](https://mlsanity.com/paper/2506.00297/) (2025)
- [ProteinZero: Self-Improving Protein Generation via Online Reinforcement Learning](https://mlsanity.com/paper/2506.07459/) (2025)
- [Protein Inverse Folding From Structure Feedback](https://mlsanity.com/paper/2506.03028/) (2025)
- [Entropy Across the Bridge: Conditional-Marginal Discretization for Flow and Schr"odinger Samplers](https://mlsanity.com/paper/2605.16126/) (2026)

## Abstract (arXiv)

Inverse folding models play an important role in structure-based design by predicting amino acid sequences that fold into desired reference structures. Models like ProteinMPNN, a message-passing encoder-decoder model, are trained to reliably produce new sequences from a reference structure. However, when applied to peptides, these models are prone to generating repetitive sequences that do not fold into the reference structure. To address this, we fine-tune ProteinMPNN to produce diverse and structurally consistent peptide sequences via Direct Preference Optimization (DPO). We derive two enhancements to DPO: online diversity regularization and domain-specific priors. Additionally, we develop a new understanding on improving diversity in decoder models. When conditioned on OpenFold generated structures, our fine-tuned models achieve state-of-the-art structural similarity scores, improving base ProteinMPNN by at least 8%. Compared to standard DPO, our regularized method achieves up to 20% higher sequence diversity with no loss in structural similarity score.