# Fold2Seq: A Joint Sequence(1D)-Fold(3D) Embedding-based Generative Model for Protein Design

arXiv: https://arxiv.org/abs/2106.13058 · HTML: https://mlsanity.com/paper/2106.13058/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 50 references that are in the map.
- First posted on arXiv 24 Jun 2021.

## Builds on (most cited first, 11 of 11)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Hierarchical Neural Story Generation](https://mlsanity.com/paper/1805.04833/) (2018)
- [Objects that Sound](https://mlsanity.com/paper/1712.06651/) (2017)
- [Zero-Shot Learning Through Cross-Modal Transfer](https://mlsanity.com/paper/1301.3666/) (2013)
- [Text2Shape: Generating Shapes from Natural Language by Learning Joint Embeddings](https://mlsanity.com/paper/1803.08495/) (2018)
- [ProGen: Language Modeling for Protein Generation](https://mlsanity.com/paper/2004.03497/) (2020)
- [Joint Embedding of Words and Labels for Text Classification](https://mlsanity.com/paper/1805.04174/) (2018)
- [Look, Imagine and Match: Improving Textual-Visual Cross-Modal Retrieval with Generative Models](https://mlsanity.com/paper/1711.06420/) (2017)
- [Zero-Shot Learning via Joint Latent Similarity Embedding](https://mlsanity.com/paper/1511.04512/) (2015)
- [CM-GANs: Cross-modal Generative Adversarial Networks for Common Representation Learning](https://mlsanity.com/paper/1710.05106/) (2017)

## Built on by (most cited first, 23 of 23)

- [Protein Representation Learning by Geometric Structure Pretraining](https://mlsanity.com/paper/2203.06125/) (2022)
- [Iterative Refinement Graph Neural Network for Antibody Sequence-Structure Co-design](https://mlsanity.com/paper/2110.04624/) (2021)
- [PiFold: Toward effective and efficient protein inverse folding](https://mlsanity.com/paper/2209.12643/) (2022)
- [Conditional Antibody Design as 3D Equivariant Graph Translation](https://mlsanity.com/paper/2208.06073/) (2022)
- [Learning Hierarchical Protein Representations via Complete 3D Graph Networks](https://mlsanity.com/paper/2207.12600/) (2022)
- [Reprogramming Pretrained Language Models for Antibody Sequence Infilling](https://mlsanity.com/paper/2210.07144/) (2022)
- [Protein Large Language Models: A Comprehensive Survey](https://mlsanity.com/paper/2502.17504/) (2025)
- [Protein Language Models and Structure Prediction: Connection and Progression](https://mlsanity.com/paper/2211.16742/) (2022)
- [AntBO: Towards Real-World Automated Antibody Design with Combinatorial Bayesian Optimisation](https://mlsanity.com/paper/2201.12570/) (2022)
- [Generative De Novo Protein Design with Global Context](https://mlsanity.com/paper/2204.10673/) (2022)
- [Antibody-Antigen Docking and Design via Hierarchical Equivariant Refinement](https://mlsanity.com/paper/2207.06616/) (2022)
- [AbODE: Ab Initio Antibody Design using Conjoined ODEs](https://mlsanity.com/paper/2306.01005/) (2023)
- [De novo protein design using geometric vector field networks](https://mlsanity.com/paper/2310.11802/) (2023)
- [Knowledge-Design: Pushing the Limit of Protein Design via Knowledge Refinement](https://mlsanity.com/paper/2305.15151/) (2023)
- [Benchmarking deep generative models for diverse antibody sequence design](https://mlsanity.com/paper/2111.06801/) (2021)
- [Clustering for Protein Representation Learning](https://mlsanity.com/paper/2404.00254/) (2024)
- [Large Language Models in Bioinformatics: A Survey](https://mlsanity.com/paper/2503.04490/) (2025)
- [Incorporating Pre-training Paradigm for Antibody Sequence-Structure Co-design](https://mlsanity.com/paper/2211.08406/) (2022)
- [Large Language Models for Bioinformatics](https://mlsanity.com/paper/2501.06271/) (2025)
- [Efficiently Predicting Protein Stability Changes Upon Single-point Mutation with Large Language Models](https://mlsanity.com/paper/2312.04019/) (2023)
- [AlphaFold Distillation for Protein Design](https://mlsanity.com/paper/2210.03488/) (2022)
- [TaxDiff: Taxonomic-Guided Diffusion Model for Protein Sequence Generation](https://mlsanity.com/paper/2402.17156/) (2024)
- [AlphaFold Database Debiasing for Robust Inverse Folding](https://mlsanity.com/paper/2506.08365/) (2025)

## Abstract (arXiv)

Designing novel protein sequences for a desired 3D topological fold is a fundamental yet non-trivial task in protein engineering. Challenges exist due to the complex sequence--fold relationship, as well as the difficulties to capture the diversity of the sequences (therefore structures and functions) within a fold. To overcome these challenges, we propose Fold2Seq, a novel transformer-based generative framework for designing protein sequences conditioned on a specific target fold. To model the complex sequence--structure relationship, Fold2Seq jointly learns a sequence embedding using a transformer and a fold embedding from the density of secondary structural elements in 3D voxels. On test sets with single, high-resolution and complete structure inputs for individual folds, our experiments demonstrate improved or comparable performance of Fold2Seq in terms of speed, coverage, and reliability for sequence design, when compared to existing state-of-the-art methods that include data-driven deep generative models and physics-based RosettaDesign. The unique advantages of fold-based Fold2Seq, in comparison to a structure-based deep model and RosettaDesign, become more evident on three additional real-world challenges originating from low-quality, incomplete, or ambiguous input structures. Source code and data are available at https://github.com/IBM/fold2seq.