# Protein Design with Dynamic Protein Vocabulary

arXiv: https://arxiv.org/abs/2505.18966 · HTML: https://mlsanity.com/paper/2505.18966/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 56 references that are in the map.
- First posted on arXiv 25 May 2025.

## Builds on (most cited first, 24 of 24)

- [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Improved Baselines with Visual Instruction Tuning](https://mlsanity.com/paper/2310.03744/) (2023)
- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [Billion-scale similarity search with GPUs](https://mlsanity.com/paper/1702.08734/) (2017)
- [Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing](https://mlsanity.com/paper/2007.15779/) (2020)
- [Neural Text Generation with Unlikelihood Training](https://mlsanity.com/paper/1908.04319/) (2019)
- [Generative Flows on Discrete State-Spaces: Enabling Multimodal Flows with Applications to Protein Co-Design](https://mlsanity.com/paper/2402.04997/) (2024)
- [ProGen2: Exploring the Boundaries of Protein Language Models](https://mlsanity.com/paper/2206.13517/) (2022)
- [Mol-Instructions: A Large-Scale Biomolecular Instruction Dataset for Large Language Models](https://mlsanity.com/paper/2306.08018/) (2023)
- [ProGen: Language Modeling for Protein Generation](https://mlsanity.com/paper/2004.03497/) (2020)
- [Diffusion Language Models Are Versatile Protein Learners](https://mlsanity.com/paper/2402.18567/) (2024)
- [Iterative Refinement Graph Neural Network for Antibody Sequence-Structure Co-design](https://mlsanity.com/paper/2110.04624/) (2021)
- [A Text-guided Protein Design Framework](https://mlsanity.com/paper/2302.04611/) (2023)
- [DPLM-2: A Multimodal Diffusion Protein Language Model](https://mlsanity.com/paper/2410.13782/) (2024)
- [Improved motif-scaffolding with SE(3) flow matching](https://mlsanity.com/paper/2401.04082/) (2024)
- [RITA: a Study on Scaling Up Generative Protein Sequence Models](https://mlsanity.com/paper/2205.05789/) (2022)
- [Antigen-Specific Antibody Design via Direct Energy-based Preference Optimization](https://mlsanity.com/paper/2403.16576/) (2024)
- [Copy Is All You Need](https://mlsanity.com/paper/2307.06962/) (2023)
- [Retrieval is Accurate Generation](https://mlsanity.com/paper/2402.17532/) (2024)
- [Annotation-guided Protein Design with Multi-Level Domain Alignment](https://mlsanity.com/paper/2404.16866/) (2024)
- [EvoLlama: Enhancing LLMs' Understanding of Proteins via Multimodal Structure and Sequence Representations](https://mlsanity.com/paper/2412.11618/) (2024)
- [Generation with Dynamic Vocabulary](https://mlsanity.com/paper/2410.08481/) (2024)

## Built on by (most cited first, 5 of 5)

- [TeleMem: Building Long-Term and Multimodal Memory for Agentic AI](https://mlsanity.com/paper/2601.06037/) (2025)
- [ProtoCycle: Reflective Tool-Augmented Planning for Text-Guided Protein Design](https://mlsanity.com/paper/2604.16896/) (2026)
- [AMix-2: Establishing Protein as a Native Modality in Large Language Models](https://mlsanity.com/paper/2605.30963/) (2026)
- [Co-Generative De Novo Functional Protein Design](https://mlsanity.com/paper/2605.00948/) (2026)
- [ProteinOPD: Towards Effective and Efficient Preference Alignment for Protein Design](https://mlsanity.com/paper/2605.10189/) (2026)

## Abstract (arXiv)

Protein design is a fundamental challenge in biotechnology, aiming to design novel sequences with specific functions within the vast space of possible proteins. Recent advances in deep generative models have enabled function-based protein design from textual descriptions, yet struggle with structural plausibility. Inspired by classical protein design methods that leverage natural protein structures, we explore whether incorporating fragments from natural proteins can enhance foldability in generative models. Our empirical results show that even random incorporation of fragments improves foldability. Building on this insight, we introduce ProDVa, a novel protein design approach that integrates a text encoder for functional descriptions, a protein language model for designing proteins, and a fragment encoder to dynamically retrieve protein fragments based on textual functional descriptions. Experimental results demonstrate that our approach effectively designs protein sequences that are both functionally aligned and structurally plausible. Compared to state-of-the-art models, ProDVa achieves comparable function alignment using less than 0.04% of the training data, while designing significantly more well-folded proteins, with the proportion of proteins having pLDDT above 70 increasing by 7.38% and those with PAE below 10 increasing by 9.6%.