# Two-Stage Fine-Tuning for Protein Sequence Generation with Targeted Amino-Acid Composition

arXiv: https://arxiv.org/abs/2606.27939 · HTML: https://mlsanity.com/paper/2606.27939/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 7 of its 20 references that are in the map.
- First posted on arXiv 26 Jun 2026.

## Builds on (most cited first, 7 of 7)

- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [Don't Stop Pretraining: Adapt Language Models to Domains and Tasks](https://mlsanity.com/paper/2004.10964/) (2020)
- [Advantage-Weighted Regression: Simple and Scalable Off-Policy Reinforcement Learning](https://mlsanity.com/paper/1910.00177/) (2019)
- [ProGen2: Exploring the Boundaries of Protein Language Models](https://mlsanity.com/paper/2206.13517/) (2022)
- [RITA: a Study on Scaling Up Generative Protein Sequence Models](https://mlsanity.com/paper/2205.05789/) (2022)
- [Reinforcement Learning for Sequence Design Leveraging Protein Language Models](https://mlsanity.com/paper/2407.03154/) (2024)
- [A deep reinforcement learning platform for antibiotic discovery](https://mlsanity.com/paper/2509.18153/) (2025)

## Abstract (arXiv)

Protein language models are standard priors for biological sequence generation, but steering them toward explicit distributional design targets remains largely unexplored. We study a constrained protein generation problem in which sequences must match a desired amino-acid (AA) composition profile while preserving plausible sequence statistics and diversity. The motivating application is synthetic feed protein design, where the AA composition of dietary proteins directly determines their nutritional value. We propose a two-stage pipeline in which domain-adaptive fine-tuning (FT) on an in-domain protein dataset is followed by iterative reward-weighted FT via reinforcement learning (RL) anchored against the FT model as a frozen reference. We evaluate the pipeline on two AA compositions and find that FT brings the average composition close to the target, while the subsequent RL enforces specific sequence constraints that FT alone cannot satisfy. We additionally evaluate the design choices of the proposed composition reward term against two baselines and an ablated variant, isolate the contribution of each training stage, and verify that AA composition alignment is achieved without degrading sequence quality.