# Peptide-GPT: Generative Design of Peptides using Generative Pre-trained Transformers and Bio-informatic Supervision

arXiv: https://arxiv.org/abs/2410.19222 · HTML: https://mlsanity.com/paper/2410.19222/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 37 references that are in the map.
- First posted on arXiv 25 Oct 2024.

## Builds on (most cited first, 8 of 8)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Defending Against Neural Fake News](https://mlsanity.com/paper/1905.12616/) (2019)
- [Molecular Transformer - A Model for Uncertainty-Calibrated Chemical Reaction Prediction](https://mlsanity.com/paper/1811.02633/) (2018)
- [Mapping the Space of Chemical Reactions Using Attention-Based Neural Networks](https://mlsanity.com/paper/2012.06051/) (2020)
- [PeptideBERT: A Language Model based on Transformers for Peptide Property Prediction](https://mlsanity.com/paper/2309.03099/) (2023)
- [Classifying Antimicrobial and Multifunctional Peptides with Bayesian Network Models](https://mlsanity.com/paper/1804.06327/) (2018)
- [GPCR-BERT: Interpreting Sequential Design of G Protein Coupled Receptors Using Protein Language Models](https://mlsanity.com/paper/2310.19915/) (2023)

## Built on by (most cited first, 5 of 5)

- [AdditiveLLM: Large Language Models Predict Defects in Additive Manufacturing](https://mlsanity.com/paper/2501.17784/) (2025)
- [Pepti-Agent: An AI Agent for Peptide Design and Optimization](https://mlsanity.com/paper/2606.15422/) (2026)
- [Energy Efficient Protein Language Models: Leveraging Small Language Models with LoRA for Controllable Protein Generation](https://mlsanity.com/paper/2411.05966/) (2024)
- [Latent Retrieval Augmented Generation of Cross-Domain Protein Binders](https://mlsanity.com/paper/2510.10480/) (2025)
- [PepEVOLVE: Position-Aware Dynamic Peptide Optimization via Group-Relative Advantage](https://mlsanity.com/paper/2511.16912/) (2025)

## Abstract (arXiv)

In recent years, natural language processing (NLP) models have demonstrated remarkable capabilities in various domains beyond traditional text generation. In this work, we introduce PeptideGPT, a protein language model tailored to generate protein sequences with distinct properties: hemolytic activity, solubility, and non-fouling characteristics. To facilitate a rigorous evaluation of these generated sequences, we established a comprehensive evaluation pipeline consisting of ideas from bioinformatics to retain valid proteins with ordered structures. First, we rank the generated sequences based on their perplexity scores, then we filter out those lying outside the permissible convex hull of proteins. Finally, we predict the structure using ESMFold and select the proteins with pLDDT values greater than 70 to ensure ordered structure. The properties of generated sequences are evaluated using task-specific classifiers - PeptideBERT and HAPPENN. We achieved an accuracy of 76.26% in hemolytic, 72.46% in non-hemolytic, 78.84% in non-fouling, and 68.06% in solubility protein generation. Our experimental results demonstrate the effectiveness of PeptideGPT in de novo protein design and underscore the potential of leveraging NLP-based approaches for paving the way for future innovations and breakthroughs in synthetic biology and bioinformatics. Codes, models, and data used in this study are freely available at: https://github.com/aayush-shah14/PeptideGPT.