mlsanityData to 7 Oct 2026

FoldToken: Learning Protein Language via Vector Quantization and Beyond

arXiv 2403.09673 ↗First posted 4 Feb 2024q-bio.BM cs.AI cs.LGcited by 21

Not on the map. Why?

Abstract (arXiv)

Is there a foreign language describing protein sequences and structures simultaneously? Protein structures, represented by continuous 3D points, have long posed a challenge due to the contrasting modeling paradigms of discrete sequences. We introduce FoldTokenizer to represent protein sequence-structure as discrete symbols. This innovative approach involves projecting residue types and structures into a discrete space, guided by a reconstruction loss for information preservation. We refer to the learned discrete symbols as FoldToken, and the sequence of FoldTokens serves as a new protein language, transforming the protein sequence-structure into a unified modality. We apply the created protein language on general backbone inpainting and antibody design tasks, building the first GPT-style model (FoldGPT) for sequence-structure co-generation with promising results. Key to our success is the substantial enhancement of the vector quantization module, Soft Conditional Vector Quantization (SoftCVQ).

Key facts. Cited by 21 papers in the map (data through 7 Oct 2026). Builds on 30 of its 50 references that are in the map. First posted on arXiv 4 Feb 2024.

Builds on

30 of 50 references.

References · 30

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2017Neural Discrete Representation Learningcited by 4,650
  4. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  5. 2021SoundStream: An End-to-End Neural Audio Codeccited by 756
  6. 2022Autoregressive Image Generation using Residual Quantizationcited by 706
  7. 2021Vector-quantized Image Modeling with Improved VQGANcited by 656
  8. 2023Language Model Beats Diffusion -- Tokenizer is Key to Visual Generationcited by 633
  9. 2023Finite Scalar Quantization: VQ-VAE Made Simplecited by 425
  10. 2020Learning from Protein Structure with Geometric Vector Perceptronscited by 308
  11. 2022Diffusion probabilistic modeling of protein backbones in 3D for the motif-scaffolding problemcited by 229
  12. 2022Protein Representation Learning by Geometric Structure Pretrainingcited by 161
18 more
  1. 2022Protein structure generation via folding diffusioncited by 113
  2. 2021Iterative Refinement Graph Neural Network for Antibody Sequence-Structure Co-designcited by 82
  3. 2022PiFold: Toward effective and efficient protein inverse foldingcited by 82
  4. 2022Conditional Antibody Design as 3D Equivariant Graph Translationcited by 75
  5. 2022AlphaDesign: A graph protein design method and benchmark on AlphaFoldDBcited by 39
  6. 2019Differentiable Product Quantization for End-to-End Embedding Compressioncited by 36
  7. 2022Image Compression with Product Quantized Masked Image Modelingcited by 32
  8. 2022Protein Sequence and Structure Co-Design with Equivariant Translationcited by 31
  9. 2014Stacked Quantizers for Compositional Vector Compressioncited by 30
  10. 2022Regression as Classification: Influence of Task Formulation on Neural Network Featurescited by 29
  11. 2023A step towards understanding why classification helps regressioncited by 15
  12. 2023Knowledge-Design: Pushing the Limit of Protein Design via Knowledge Refinementcited by 11
  13. 2023DiffSDS: A language diffusion model for protein backbone inpainting under geometric conditions and constraintscited by 10
  14. 2023Cross-Gate MLP with Protein Complex Invariant Embedding is A One-Shot Antibody Designercited by 8
  15. 2023Progressive Multi-Modality Learning for Inverse Protein Foldingcited by 5
  16. 2023Streaming Active Learning for Regression Problems Using Regression via Classificationcited by 2
  17. 2023Functional Geometry Guided Protein Sequence and Backbone Structure Co-Designcited by 2
  18. 2023Efficiently Predicting Protein Stability Changes Upon Single-point Mutation with Large Language Modelscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 9; 2025: 3; 2026: 9.

21 papers.

Sort:

By area:

  1. 2024Addressing Representation Collapse in Vector Quantized Models with One Linear Layercited by 77
  2. 2024DPLM-2: A Multimodal Diffusion Protein Language Modelcited by 59
  3. 2025Token Reduction Should Go Beyond Efficiency in Generative Models -- From Vision, Language to Multimodalitycited by 49
  4. 2024Leveraging Biomolecule and Natural Language through Multi-Modal Learning: A Surveycited by 16
  5. 2024Structure Language Models for Protein Conformation Generationcited by 16
  6. 2024Learning the Language of Protein Structurecited by 9
  7. 2024Bio2Token: All-atom tokenization of any biomolecular structure with Mambacited by 5
  8. 2025Flow Autoencoders are Effective Protein Tokenizerscited by 4
  9. 2024Constraint Decoupled Latent Diffusion for Protein Backmappingcited by 2
  10. 2026Adaptive Protein Tokenizationcited by 2
  11. 2024FoldToken2: Learning compact, invariant and generative protein structure languagecited by 1
  12. 2024Tokenizing 3D Molecule Structure with Quantized Spherical Coordinatescited by 1
9 more
  1. 2026VecFormer: Towards Efficient and Generalizable Graph Transformer with Graph Token Attentioncited by 1
  2. 2026PaperFit: Vision-in-the-Loop Typesetting Optimization for Scientific Documentscited by 1
  3. 2025AlphaFold Database Debiasing for Robust Inverse Foldingcited by 0
  4. 2026Reinforcement-Learned Unequal Error Protection for Quantized Semantic Embeddingscited by 0
  5. 2026RiboSphere: Learning Unified and Efficient Representations of RNA Structurescited by 0
  6. 2026Mitigating Premature Discretization with Progressive Quantization for Robust Vector Tokenizationcited by 0
  7. 2026Yeti: A compact protein structure tokenizer for reconstruction and multi-modal generationcited by 0
  8. 2026Discrete Diffusion Models: A Unified Framework from Tokenization to Generationcited by 0
  9. 2026Flow-Matching-Based Protein Structure Tokenizer Made Efficient and Easycited by 0