# Atom-level Protein Representation Learning Improves Protein Structure Prediction

arXiv: https://arxiv.org/abs/2605.22133 · HTML: https://mlsanity.com/paper/2605.22133/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 38 references that are in the map.
- First posted on arXiv 21 May 2026.

## Builds on (most cited first, 11 of 11)

- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Models](https://mlsanity.com/paper/2308.06721/) (2023)
- [Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Think](https://mlsanity.com/paper/2410.06940/) (2024)
- [REPA-E: Unlocking VAE for End-to-End Tuning with Latent Diffusion Transformers](https://mlsanity.com/paper/2504.10483/) (2025)
- [Return of Unconditional Generation: A Self-supervised Representation Generation Method](https://mlsanity.com/paper/2312.03701/) (2023)
- [Learning to design protein-protein interactions with enhanced generalization](https://mlsanity.com/paper/2310.18515/) (2023)
- [SimpleFold: Folding Proteins is Simpler than You Think](https://mlsanity.com/paper/2509.18480/) (2025)
- [MC-BERT: Efficient Language Pre-Training via a Meta Controller](https://mlsanity.com/paper/2006.05744/) (2020)
- [Protein Structure Tokenization: Benchmarking and New Recipe](https://mlsanity.com/paper/2503.00089/) (2025)
- [Test-Time Conditioning with Representation-Aligned Visual Features](https://mlsanity.com/paper/2602.03753/) (2026)

## Abstract (arXiv)

Recent advances in generative modeling show that pretrained representations can improve generation as conditioning features or alignment targets. Motivated by this, we study protein representations for predicting structures beyond conventional function annotation. We propose TriProRep, a structure-aware pretraining method that jointly models three aligned residue-level views: amino-acid identity, backbone geometry, and local full-atom geometry, discretely encoded via VQ-VAE tokenizers. By pretraining to recover original tokens from generator-corrupted views, TriProRep learns to distinguish plausible but incorrect cross-view augmentations from the original protein. We further introduce RepSP, a benchmark for evaluating protein representations in structure-predictive settings. RepSP tests three uses of representations: homodimer co-folding from apo-chain representations, residue-level prediction of homodimer-derived interaction properties, and representation-aligned monomer structure prediction. Across these tasks, TriProRep improves over sequence-only and prior structure-aware representation models, while maintaining competitive performance on conventional benchmarks.