# Multi-Scale Representation Learning on Proteins

arXiv: https://arxiv.org/abs/2204.02337 · HTML: https://mlsanity.com/paper/2204.02337/

## Key facts

- Cited by 52 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 69 references that are in the map.
- First posted on arXiv 4 Apr 2022.

## Builds on (most cited first, 16 of 16)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Neural Message Passing for Quantum Chemistry](https://mlsanity.com/paper/1704.01212/) (2017)
- [Fast Graph Representation Learning with PyTorch Geometric](https://mlsanity.com/paper/1903.02428/) (2019)
- [Evaluating Protein Transfer Learning with TAPE](https://mlsanity.com/paper/1906.08230/) (2019)
- [Hierarchical Generation of Molecular Graphs using Structural Motifs](https://mlsanity.com/paper/2002.03230/) (2020)
- [DeepDTA: Deep Drug-Target Binding Affinity Prediction](https://mlsanity.com/paper/1801.10193/) (2018)
- [ProtTrans: Towards Cracking the Language of Life's Code Through Self-Supervised Deep Learning and High Performance Computing](https://mlsanity.com/paper/2007.06225/) (2020)
- [ATOM3D: Tasks On Molecules in Three Dimensions](https://mlsanity.com/paper/2012.04035/) (2020)
- [Protein-Ligand Scoring with Convolutional Neural Networks](https://mlsanity.com/paper/1612.02751/) (2016)
- [Superpixels: An Evaluation of the State-of-the-Art](https://mlsanity.com/paper/1612.01601/) (2016)
- [Learning protein sequence embeddings using information from structure](https://mlsanity.com/paper/1902.08661/) (2019)
- [DeepSF: deep convolutional neural network for mapping protein sequences to folds](https://mlsanity.com/paper/1706.01010/) (2017)
- [End-to-End Learning on 3D Protein Structure for Interface Prediction](https://mlsanity.com/paper/1807.01297/) (2018)
- [Deriving Neural Architectures from Sequence and Graph Kernels](https://mlsanity.com/paper/1705.09037/) (2017)
- [Improving detection of protein-ligand binding sites with 3D segmentation](https://mlsanity.com/paper/1904.06517/) (2019)

## Built on by (most cited first, 30 of 52)

- [DiffDock: Diffusion Steps, Twists, and Turns for Molecular Docking](https://mlsanity.com/paper/2210.01776/) (2022)
- [3D Equivariant Diffusion for Target-Aware Molecule Generation and Affinity Prediction](https://mlsanity.com/paper/2303.03543/) (2023)
- [Learning Hierarchical Protein Representations via Complete 3D Graph Networks](https://mlsanity.com/paper/2207.12600/) (2022)
- [Multimodal learning with graphs](https://mlsanity.com/paper/2209.03299/) (2022)
- [Structure-based drug design with geometric deep learning](https://mlsanity.com/paper/2210.11250/) (2022)
- [Structure-aware Protein Self-supervised Learning](https://mlsanity.com/paper/2204.04213/) (2022)
- [Contrastive Representation Learning for 3D Protein Structures](https://mlsanity.com/paper/2205.15675/) (2022)
- [DiffPack: A Torsional Diffusion Model for Autoregressive Protein Side-Chain Packing](https://mlsanity.com/paper/2306.01794/) (2023)
- [Generalist Equivariant Transformer Towards 3D Molecular Interaction Learning](https://mlsanity.com/paper/2306.01474/) (2023)
- [Unsupervised Protein-Ligand Binding Energy Prediction via Neural Euler's Rotation Equation](https://mlsanity.com/paper/2301.10814/) (2023)
- [Geometry-Complete Perceptron Networks for 3D Molecular Graphs](https://mlsanity.com/paper/2211.02504/) (2022)
- [Evaluating representation learning on the protein structure universe](https://mlsanity.com/paper/2406.13864/) (2024)
- [Antibody-Antigen Docking and Design via Hierarchical Equivariant Refinement](https://mlsanity.com/paper/2207.06616/) (2022)
- [Integration of Pre-trained Protein Language Models into Geometric Deep Learning Networks](https://mlsanity.com/paper/2212.03447/) (2022)
- [Protein-ligand binding representation learning from fine-grained interactions](https://mlsanity.com/paper/2311.16160/) (2023)
- [Pre-training with Fractional Denoising to Enhance Molecular Property Prediction](https://mlsanity.com/paper/2407.11086/) (2024)
- [Pre-Training Protein Encoder via Siamese Sequence-Structure Diffusion Trajectory Prediction](https://mlsanity.com/paper/2301.12068/) (2023)
- [ProFSA: Self-supervised Pocket Pretraining via Protein Fragment-Surroundings Alignment](https://mlsanity.com/paper/2310.07229/) (2023)
- [Causal Optimal Transport of Abstractions](https://mlsanity.com/paper/2312.08107/) (2023)
- [Learning Harmonic Molecular Representations on Riemannian Manifold](https://mlsanity.com/paper/2303.15520/) (2023)
- [Knowledge-augmented Graph Machine Learning for Drug Discovery: A Survey](https://mlsanity.com/paper/2302.08261/) (2023)
- [Bi-level Contrastive Learning for Knowledge-Enhanced Molecule Representations](https://mlsanity.com/paper/2306.01631/) (2023)
- [AtomSurf : Surface Representation for Learning on Protein Structures](https://mlsanity.com/paper/2309.16519/) (2023)
- [An Equivariant Pretrained Transformer for Unified 3D Molecular Representation Learning](https://mlsanity.com/paper/2402.12714/) (2024)
- [A Cross-Field Fusion Strategy for Drug-Target Interaction Prediction](https://mlsanity.com/paper/2405.14545/) (2024)
- [GLProtein: Global-and-Local Structure Aware Protein Representation Learning](https://mlsanity.com/paper/2506.06294/) (2025)
- [Modeling Molecular Structures with Intrinsic Diffusion Models](https://mlsanity.com/paper/2302.12255/) (2023)
- [Self-Conditioned Denoising for Atomistic Representation Learning](https://mlsanity.com/paper/2603.17196/) (2026)
- [EGR: Equivariant Graph Refinement and Assessment of 3D Protein Complex Structures](https://mlsanity.com/paper/2205.10390/) (2022)
- [Holographic-(V)AE: an end-to-end SO(3)-Equivariant (Variational) Autoencoder in Fourier Space](https://mlsanity.com/paper/2209.15567/) (2022)

## Abstract (arXiv)

Proteins are fundamental biological entities mediating key roles in cellular function and disease. This paper introduces a multi-scale graph construction of a protein -- HoloProt -- connecting surface to structure and sequence. The surface captures coarser details of the protein, while sequence as primary component and structure -- comprising secondary and tertiary components -- capture finer details. Our graph encoder then learns a multi-scale representation by allowing each level to integrate the encoding from level(s) below with the graph at that level. We test the learned representation on different tasks, (i.) ligand binding affinity (regression), and (ii.) protein function prediction (classification). On the regression task, contrary to previous methods, our model performs consistently and reliably across different dataset splits, outperforming all baselines on most splits. On the classification task, it achieves a performance close to the top-performing model while using 10x fewer parameters. To improve the memory efficiency of our construction, we segment the multiplex protein surface manifold into molecular superpixels and substitute the surface with these superpixels at little to no performance loss.