# Evaluating representation learning on the protein structure universe

arXiv: https://arxiv.org/abs/2406.13864 · HTML: https://mlsanity.com/paper/2406.13864/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 39 of its 91 references that are in the map.
- First posted on arXiv 19 Jun 2024.

## Builds on (most cited first, 30 of 39)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Fast Graph Representation Learning with PyTorch Geometric](https://mlsanity.com/paper/1903.02428/) (2019)
- [Datasheets for Datasets](https://mlsanity.com/paper/1803.09010/) (2018)
- [E(n) Equivariant Graph Neural Networks](https://mlsanity.com/paper/2102.09844/) (2021)
- [Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point clouds](https://mlsanity.com/paper/1802.08219/) (2018)
- [Benchmarking Graph Neural Networks](https://mlsanity.com/paper/2003.00982/) (2020)
- [Directional Message Passing for Molecular Graphs](https://mlsanity.com/paper/2003.03123/) (2020)
- [MACE: Higher Order Equivariant Message Passing Neural Networks for Fast and Accurate Force Fields](https://mlsanity.com/paper/2206.07697/) (2022)
- [Equivariant message passing for the prediction of tensorial properties and molecular spectra](https://mlsanity.com/paper/2102.03150/) (2021)
- [Learning from Protein Structure with Geometric Vector Perceptrons](https://mlsanity.com/paper/2009.01411/) (2020)
- [DiffDock: Diffusion Steps, Twists, and Turns for Molecular Docking](https://mlsanity.com/paper/2210.01776/) (2022)
- [Therapeutics Data Commons: Machine Learning Datasets and Tasks for Drug Discovery and Development](https://mlsanity.com/paper/2102.09548/) (2021)
- [Evaluating Protein Transfer Learning with TAPE](https://mlsanity.com/paper/1906.08230/) (2019)
- [Structure-based Drug Design with Equivariant Diffusion Models](https://mlsanity.com/paper/2210.13695/) (2022)
- [Design Space for Graph Neural Networks](https://mlsanity.com/paper/2011.08843/) (2020)
- [e3nn: Euclidean Neural Networks](https://mlsanity.com/paper/2207.09453/) (2022)
- [Protein Representation Learning by Geometric Structure Pretraining](https://mlsanity.com/paper/2203.06125/) (2022)
- [Pre-training via Denoising for Molecular Property Prediction](https://mlsanity.com/paper/2206.00133/) (2022)
- [On the Expressive Power of Geometric Graph Neural Networks](https://mlsanity.com/paper/2301.09308/) (2023)
- [Iterative Refinement Graph Neural Network for Antibody Sequence-Structure Co-design](https://mlsanity.com/paper/2110.04624/) (2021)
- [PEER: A Comprehensive and Multi-Task Benchmark for Protein Sequence Understanding](https://mlsanity.com/paper/2206.02096/) (2022)
- [ATOM3D: Tasks On Molecules in Three Dimensions](https://mlsanity.com/paper/2012.04035/) (2020)
- [Molecular Geometry Pretraining with SE(3)-Invariant Denoising Distance Matching](https://mlsanity.com/paper/2206.13602/) (2022)
- [A Hitchhiker's Guide to Geometric GNNs for 3D Atomic Systems](https://mlsanity.com/paper/2312.07511/) (2023)
- [Learning Hierarchical Protein Representations via Complete 3D Graph Networks](https://mlsanity.com/paper/2207.12600/) (2022)
- [Multi-Scale Representation Learning on Proteins](https://mlsanity.com/paper/2204.02337/) (2022)
- [DeepSF: deep convolutional neural network for mapping protein sequences to folds](https://mlsanity.com/paper/1706.01010/) (2017)
- [TorchDrug: A Powerful and Flexible Machine Learning Platform for Drug Discovery](https://mlsanity.com/paper/2202.08320/) (2022)
- [Simple GNN Regularisation for 3D Molecular Property Prediction & Beyond](https://mlsanity.com/paper/2106.07971/) (2021)

## Built on by (most cited first, 16 of 16)

- [Protein Structure Tokenization: Benchmarking and New Recipe](https://mlsanity.com/paper/2503.00089/) (2025)
- [Using pretrained graph neural networks with token mixers as geometric featurizers for conformational dynamics](https://mlsanity.com/paper/2409.19838/) (2024)
- [Multi-state Protein Design with DynamicMPNN](https://mlsanity.com/paper/2507.21938/) (2025)
- [Topotein: Topological Deep Learning for Protein Representation Learning](https://mlsanity.com/paper/2509.03885/) (2025)
- [A Comprehensive Benchmark for RNA 3D Structure-Function Modeling](https://mlsanity.com/paper/2503.21681/) (2025)
- [Fast and Interpretable Protein Substructure Alignment via Optimal Transport](https://mlsanity.com/paper/2510.11752/) (2025)
- [Hierarchical geometric deep learning enables scalable analysis of molecular dynamics](https://mlsanity.com/paper/2512.06520/) (2025)
- [Geometric Self-Supervised Pretraining on 3D Protein Structures using Subgraphs](https://mlsanity.com/paper/2406.14142/) (2024)
- [General Protein Pretraining or Domain-Specific Designs? Benchmarking Protein Modeling on Realistic Applications](https://mlsanity.com/paper/2506.02052/) (2025)
- [Demystify Protein Generation with Hierarchical Conditional Diffusion Models](https://mlsanity.com/paper/2507.18603/) (2025)
- [Multi-Scale Protein Structure Modelling with Geometric Graph U-Nets](https://mlsanity.com/paper/2512.06752/) (2025)
- [MMPG: MoE-based Adaptive Multi-Perspective Graph Fusion for Protein Representation Learning](https://mlsanity.com/paper/2601.10157/) (2026)
- [PUFFIN: Protein Unit Discovery with Functional Supervision](https://mlsanity.com/paper/2604.14796/) (2026)
- [PRIME: Protein Representation via Physics-Informed Multiscale Equivariant Hierarchies](https://mlsanity.com/paper/2605.01625/) (2026)
- [Hyper-Fold: Exploring the Expressive Limit of Sequence-Geometry Learning for Proteins via Hypergraph Modeling](https://mlsanity.com/paper/2608.29207/) (2026)
- [Predicting Transmembrane Protein Topology from 3D Structure](https://mlsanity.com/paper/2609.30446/) (2026)

## Abstract (arXiv)

We introduce ProteinWorkshop, a comprehensive benchmark suite for representation learning on protein structures with Geometric Graph Neural Networks. We consider large-scale pre-training and downstream tasks on both experimental and predicted structures to enable the systematic evaluation of the quality of the learned structural representation and their usefulness in capturing functional relationships for downstream tasks. We find that: (1) large-scale pretraining on AlphaFold structures and auxiliary tasks consistently improve the performance of both rotation-invariant and equivariant GNNs, and (2) more expressive equivariant GNNs benefit from pretraining to a greater extent compared to invariant models. We aim to establish a common ground for the machine learning and computational biology communities to rigorously compare and advance protein structure representation learning. Our open-source codebase reduces the barrier to entry for working with large protein structure datasets by providing: (1) storage-efficient dataloaders for large-scale structural databases including AlphaFoldDB and ESM Atlas, as well as (2) utilities for constructing new tasks from the entire PDB. ProteinWorkshop is available at: github.com/a-r-j/ProteinWorkshop.