# ProTok: Unified Protein Tokenization via Modular Multi-Codebook Quantization

arXiv: https://arxiv.org/abs/2610.18481 · HTML: https://mlsanity.com/paper/2610.18481/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 18 of its 46 references that are in the map.
- First posted on arXiv 13 Oct 2026.

## Builds on (most cited first, 18 of 18)

- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Models](https://mlsanity.com/paper/2506.05176/) (2025)
- [Learning from Protein Structure with Geometric Vector Perceptrons](https://mlsanity.com/paper/2009.01411/) (2020)
- [Evaluating Protein Transfer Learning with TAPE](https://mlsanity.com/paper/1906.08230/) (2019)
- [UniTok: A Unified Tokenizer for Visual Generation and Understanding](https://mlsanity.com/paper/2502.20321/) (2025)
- [Addressing Representation Collapse in Vector Quantized Models with One Linear Layer](https://mlsanity.com/paper/2411.02038/) (2024)
- [DPLM-2: A Multimodal Diffusion Protein Language Model](https://mlsanity.com/paper/2410.13782/) (2024)
- [ProteinBench: A Holistic Evaluation of Protein Foundation Models](https://mlsanity.com/paper/2409.06744/) (2024)
- [FoldToken: Learning Protein Language via Vector Quantization and Beyond](https://mlsanity.com/paper/2403.09673/) (2024)
- [Geometry-Complete Perceptron Networks for 3D Molecular Graphs](https://mlsanity.com/paper/2211.02504/) (2022)
- [Protein Structure Tokenization: Benchmarking and New Recipe](https://mlsanity.com/paper/2503.00089/) (2025)
- [Exploration into Translation-Equivariant Image Quantization](https://mlsanity.com/paper/2112.00384/) (2021)
- [Bio2Token: All-atom tokenization of any biomolecular structure with Mamba](https://mlsanity.com/paper/2410.19110/) (2024)
- [Protein Structure Tokenization via Geometric Byte Pair Encoding](https://mlsanity.com/paper/2511.11758/) (2025)
- [Yeti: A compact protein structure tokenizer for reconstruction and multi-modal generation](https://mlsanity.com/paper/2605.09981/) (2026)
- [Flow-Matching-Based Protein Structure Tokenizer Made Efficient and Easy](https://mlsanity.com/paper/2609.33129/) (2026)

## Abstract (arXiv)

Protein sequence, structure, and functional annotations each capture complementary aspects of protein biology, yet existing tokenizers encode each modality independently, limiting the expressiveness of discrete protein representations. We present ProTok, a unified protein tokenizer that maps all three modalities into a modular discrete vocabulary via product quantization with per-modality codebook groups and modality dropout. A single ProTok checkpoint supports flexible inference from any modality subset without retraining. On seven StructTokenBench benchmarks spanning 14 tasks and 28 test splits, ProTok achieves an average rank of 1.4 across all structure-only tokenizers, placing first on 21 splits and among the top two on 27 of the 28. These gains come without sacrificing reconstruction: ProTok's tokens reach 0.939 lDDT on CATH 4.2, showing that multi-codebook quantization and multi-modal training yield tokens that are both reconstructable and informative. ProTok's structure tokens further lead all baseline structure tokenizers on zero-shot functional retrieval across CATH, Enzyme Commission, and Gene Ontology label hierarchies. The vocabulary also supports generation: a compact language model trained from scratch over ProTok tokens jointly generates backbone and sequence, with 52% of sampled backbones designable versus 21% for a parameter-matched GeoBPE baseline. ProTok shows that structure, sequence, and function can share one discrete vocabulary, motivating protein language models that reason over a unified token stream.