# ProFSA: Self-supervised Pocket Pretraining via Protein Fragment-Surroundings Alignment

arXiv: https://arxiv.org/abs/2310.07229 · HTML: https://mlsanity.com/paper/2310.07229/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 51 references that are in the map.
- First posted on arXiv 11 Oct 2023.

## Builds on (most cited first, 19 of 19)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Unsupervised Feature Learning via Non-Parametric Instance-level Discrimination](https://mlsanity.com/paper/1805.01978/) (2018)
- [Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hypersphere](https://mlsanity.com/paper/2005.10242/) (2020)
- [Strategies for Pre-training Graph Neural Networks](https://mlsanity.com/paper/1905.12265/) (2019)
- [Evaluating Protein Transfer Learning with TAPE](https://mlsanity.com/paper/1906.08230/) (2019)
- [DeepDTA: Deep Drug-Target Binding Affinity Prediction](https://mlsanity.com/paper/1801.10193/) (2018)
- [ProtTrans: Towards Cracking the Language of Life's Code Through Self-Supervised Deep Learning and High Performance Computing](https://mlsanity.com/paper/2007.06225/) (2020)
- [Pre-training via Denoising for Molecular Property Prediction](https://mlsanity.com/paper/2206.00133/) (2022)
- [ATOM3D: Tasks On Molecules in Three Dimensions](https://mlsanity.com/paper/2012.04035/) (2020)
- [Molecular Geometry Pretraining with SE(3)-Invariant Denoising Distance Matching](https://mlsanity.com/paper/2206.13602/) (2022)
- [Learning Hierarchical Protein Representations via Complete 3D Graph Networks](https://mlsanity.com/paper/2207.12600/) (2022)
- [DeepAffinity: Interpretable Deep Learning of Compound-Protein Affinity through Unified Recurrent and Convolutional Neural Networks](https://mlsanity.com/paper/1806.07537/) (2018)
- [Multi-Scale Representation Learning on Proteins](https://mlsanity.com/paper/2204.02337/) (2022)
- [Learning protein sequence embeddings using information from structure](https://mlsanity.com/paper/1902.08661/) (2019)
- [Fractional Denoising for 3D Molecular Pre-training](https://mlsanity.com/paper/2307.10683/) (2023)
- [Generalist Equivariant Transformer Towards 3D Molecular Interaction Learning](https://mlsanity.com/paper/2306.01474/) (2023)
- [CoSP: Co-supervised pretraining of pocket and ligand](https://mlsanity.com/paper/2206.12241/) (2022)

## Built on by (most cited first, 13 of 13)

- [A Survey of Geometric Graph Neural Networks: Data Structures, Models and Applications](https://mlsanity.com/paper/2403.00485/) (2024)
- [Protein-ligand binding representation learning from fine-grained interactions](https://mlsanity.com/paper/2311.16160/) (2023)
- [An Equivariant Pretrained Transformer for Unified 3D Molecular Representation Learning](https://mlsanity.com/paper/2402.12714/) (2024)
- [Multi-level Interaction Modeling for Protein Mutational Effect Prediction](https://mlsanity.com/paper/2405.17802/) (2024)
- [Self-Conditioned Denoising for Atomistic Representation Learning](https://mlsanity.com/paper/2603.17196/) (2026)
- [SIU: A Million-Scale Structural Small Molecule-Protein Interaction Dataset for Unbiased Bioactivity Prediction](https://mlsanity.com/paper/2406.08961/) (2024)
- [Integrating Protein Dynamics into Structure-Based Drug Design via Full-Atom Stochastic Flows](https://mlsanity.com/paper/2503.03989/) (2025)
- [MolChord: Structure-Sequence Alignment for Protein-Guided Drug Design](https://mlsanity.com/paper/2510.27671/) (2025)
- [Structure-guided molecular design with contrastive 3D protein-ligand learning](https://mlsanity.com/paper/2604.19562/) (2026)
- [h-MINT: Modeling Pocket-Ligand Binding with Hierarchical Molecular Interaction Network](https://mlsanity.com/paper/2604.23134/) (2026)
- [CARD: Coarse-to-fine Autoregressive Modeling with Radix-based Decomposition for Transferable Free Energy Estimation](https://mlsanity.com/paper/2605.02657/) (2026)
- [Curvature-Informed Potential Energy Surface for Protein-Ligand Binding Affinity Prediction](https://mlsanity.com/paper/2606.14217/) (2026)
- [Blending experimental electron density information to protein-ligand interaction](https://mlsanity.com/paper/2610.14862/) (2026)

## Abstract (arXiv)

Pocket representations play a vital role in various biomedical applications, such as druggability estimation, ligand affinity prediction, and de novo drug design. While existing geometric features and pretrained representations have demonstrated promising results, they usually treat pockets independent of ligands, neglecting the fundamental interactions between them. However, the limited pocket-ligand complex structures available in the PDB database (less than 100 thousand non-redundant pairs) hampers large-scale pretraining endeavors for interaction modeling. To address this constraint, we propose a novel pocket pretraining approach that leverages knowledge from high-resolution atomic protein structures, assisted by highly effective pretrained small molecule representations. By segmenting protein structures into drug-like fragments and their corresponding pockets, we obtain a reasonable simulation of ligand-receptor interactions, resulting in the generation of over 5 million complexes. Subsequently, the pocket encoder is trained in a contrastive manner to align with the representation of pseudo-ligand furnished by some pretrained small molecule encoders. Our method, named ProFSA, achieves state-of-the-art performance across various tasks, including pocket druggability prediction, pocket matching, and ligand binding affinity prediction. Notably, ProFSA surpasses other pretraining methods by a substantial margin. Moreover, our work opens up a new avenue for mitigating the scarcity of protein-ligand complex data through the utilization of high-quality and diverse protein structure databases.