# Preference optimization of protein language models as a multi-objective binder design paradigm

arXiv: https://arxiv.org/abs/2403.04187 · HTML: https://mlsanity.com/paper/2403.04187/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Preference optimization for molecular design, one of 10 directions in LLM alignment / RLHF / preference optimisation.
- Builds on 13 of its 34 references that are in the map.
- First posted on arXiv 7 Mar 2024.

## Placed in

- LLM alignment / RLHF / preference optimisation / [Preference optimization for molecular design](https://mlsanity.com/alignment/preference-optimization-for-molecular-design/)

## Builds on (most cited first, 13 of 13)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [Deep reinforcement learning from human preferences](https://mlsanity.com/paper/1706.03741/) (2017)
- [QLoRA: Efficient Finetuning of Quantized LLMs](https://mlsanity.com/paper/2305.14314/) (2023)
- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [Hierarchical Neural Story Generation](https://mlsanity.com/paper/1805.04833/) (2018)
- [ProGen2: Exploring the Boundaries of Protein Language Models](https://mlsanity.com/paper/2206.13517/) (2022)
- [Protein Design with Guided Discrete Diffusion](https://mlsanity.com/paper/2305.20009/) (2023)
- [A Multi-Modal Contrastive Diffusion Model for Therapeutic Peptide Generation](https://mlsanity.com/paper/2312.15665/) (2023)
- [Preference Optimization for Molecular Language Models](https://mlsanity.com/paper/2310.12304/) (2023)
- [Generative Antibody Design for Complementary Chain Pairing Sequences through Encoder-Decoder Language Model](https://mlsanity.com/paper/2301.02748/) (2023)
- [Pairing interacting protein sequences using masked language modeling](https://mlsanity.com/paper/2308.07136/) (2023)

## Built on by (most cited first, 4 of 4)

- [Boltzmann-Aligned Inverse Folding Model as a Predictor of Mutational Effects on Protein-Protein Interactions](https://mlsanity.com/paper/2410.09543/) (2024)
- [Improving Inverse Folding for Peptide Design with Diversity-regularized Direct Preference Optimization](https://mlsanity.com/paper/2410.19471/) (2024)
- [Protein Inverse Folding From Structure Feedback](https://mlsanity.com/paper/2506.03028/) (2025)
- [g-DPO: Scalable Preference Optimization for Protein Language Models](https://mlsanity.com/paper/2510.19474/) (2025)

## Abstract (arXiv)

We present a multi-objective binder design paradigm based on instruction fine-tuning and direct preference optimization (DPO) of autoregressive protein language models (pLMs). Multiple design objectives are encoded in the language model through direct optimization on expert curated preference sequence datasets comprising preferred and dispreferred distributions. We show the proposed alignment strategy enables ProtGPT2 to effectively design binders conditioned on specified receptors and a drug developability criterion. Generated binder samples demonstrate median isoelectric point (pI) improvements by 17%-60%.