# Bio2Token: All-atom tokenization of any biomolecular structure with Mamba

arXiv: https://arxiv.org/abs/2410.19110 · HTML: https://mlsanity.com/paper/2410.19110/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Placed in Core SSM architectures for language, one of 10 directions in State-space sequence models (S4, Mamba).
- Builds on 16 of its 36 references that are in the map.
- First posted on arXiv 24 Oct 2024.

## Placed in

- State-space sequence models (S4, Mamba) / [Core SSM architectures for language](https://mlsanity.com/ssm/core-ssm-architectures-for-language/)

## Builds on (most cited first, 16 of 16)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://mlsanity.com/paper/2312.00752/) (2023)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [Efficiently Modeling Long Sequences with Structured State Spaces](https://mlsanity.com/paper/2111.00396/) (2021)
- [Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality](https://mlsanity.com/paper/2405.21060/) (2024)
- [Equivariant Diffusion for Molecule Generation in 3D](https://mlsanity.com/paper/2203.17003/) (2022)
- [Finite Scalar Quantization: VQ-VAE Made Simple](https://mlsanity.com/paper/2309.15505/) (2023)
- [Structure-based Drug Design with Equivariant Diffusion Models](https://mlsanity.com/paper/2210.13695/) (2022)
- [Language models can generate molecules, materials, and protein binding sites directly in three dimensions as XYZ, CIF, and PDB files](https://mlsanity.com/paper/2305.05708/) (2023)
- [A Purely Algebraic Justification of the Kabsch-Umeyama Algorithm](https://mlsanity.com/paper/1902.03138/) (2019)
- [FoldToken: Learning Protein Language via Vector Quantization and Beyond](https://mlsanity.com/paper/2403.09673/) (2024)
- [BindGPT: A Scalable Framework for 3D Molecular Design via Language Modeling and Reinforcement Learning](https://mlsanity.com/paper/2406.03686/) (2024)
- [Learning the Language of Protein Structure](https://mlsanity.com/paper/2405.15840/) (2024)
- [Geometry Informed Tokenization of Molecules for Language Model Generation](https://mlsanity.com/paper/2408.10120/) (2024)
- [nabla2DFT: A Universal Quantum Chemistry Dataset of Drug-Like Molecules and a Benchmark for Neural Network Potentials](https://mlsanity.com/paper/2406.14347/) (2024)

## Built on by (most cited first, 6 of 6)

- [Protein Structure Tokenization: Benchmarking and New Recipe](https://mlsanity.com/paper/2503.00089/) (2025)
- [Scalable Autoregressive 3D Molecule Generation](https://mlsanity.com/paper/2505.13791/) (2025)
- [Flow Autoencoders are Effective Protein Tokenizers](https://mlsanity.com/paper/2510.00351/) (2025)
- [Yeti: A compact protein structure tokenizer for reconstruction and multi-modal generation](https://mlsanity.com/paper/2605.09981/) (2026)
- [ENSEMBITS: an alphabet of protein conformational ensembles](https://mlsanity.com/paper/2605.13789/) (2026)
- [ProTok: Unified Protein Tokenization via Modular Multi-Codebook Quantization](https://mlsanity.com/paper/2610.18481/) (2026)

## Abstract (arXiv)

Efficient encoding and representation of large 3D molecular structures with high fidelity is critical for biomolecular design applications. Despite this, many representation learning approaches restrict themselves to modeling smaller systems or use coarse-grained approximations of the systems, for example modeling proteins at the resolution of amino acid residues rather than at the level of individual atoms. To address this, we develop quantized auto-encoders that learn atom-level tokenizations of complete proteins, RNA and small molecule structures with reconstruction accuracies well below 1 Angstrom. We demonstrate that a simple Mamba state space model architecture is efficient compared to an SE(3)-invariant IPA architecture, reaches competitive accuracies and can scale to systems with almost 100,000 atoms. The learned structure tokens of bio2token may serve as the input for all-atom generative models in the future.