# Revealing data leakage in protein interaction benchmarks

arXiv: https://arxiv.org/abs/2404.10457 · HTML: https://mlsanity.com/paper/2404.10457/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 60 references that are in the map.
- First posted on arXiv 16 Apr 2024.

## Builds on (most cited first, 14 of 14)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [DiffDock: Diffusion Steps, Twists, and Turns for Molecular Docking](https://mlsanity.com/paper/2210.01776/) (2022)
- [Independent SE(3)-Equivariant Models for End-to-End Rigid Protein Docking](https://mlsanity.com/paper/2111.07786/) (2021)
- [End-to-End Learning on 3D Protein Structure for Interface Prediction](https://mlsanity.com/paper/1807.01297/) (2018)
- [DiffDock-PP: Rigid Protein-Protein Docking with Diffusion Models](https://mlsanity.com/paper/2304.03889/) (2023)
- [DeepRank: A New Deep Architecture for Relevance Ranking in Information Retrieval](https://mlsanity.com/paper/1710.05649/) (2017)
- [Pre-training of Graph Neural Network for Modeling Effects of Mutations on Protein-Protein Binding Affinity](https://mlsanity.com/paper/2008.12473/) (2020)
- [Geometric Transformers for Protein Interface Contact Prediction](https://mlsanity.com/paper/2110.02423/) (2021)
- [Learning to design protein-protein interactions with enhanced generalization](https://mlsanity.com/paper/2310.18515/) (2023)
- [DIPS-Plus: The Enhanced Database of Interacting Protein Structures for Interface Prediction](https://mlsanity.com/paper/2106.04362/) (2021)
- [Predicting mutational effects on protein-protein binding via a side-chain diffusion probabilistic model](https://mlsanity.com/paper/2310.19849/) (2023)
- [Rigid Protein-Protein Docking via Equivariant Elliptic-Paraboloid Interface Prediction](https://mlsanity.com/paper/2401.08986/) (2024)
- [Growing ecosystem of deep learning methods for modeling proteinunicodex2013protein interactions](https://mlsanity.com/paper/2310.06725/) (2023)

## Built on by (most cited first, 5 of 5)

- [MassSpecGym in the Wild: Uncovering and Correcting Evaluation Pitfalls in AI-Driven Molecule Discovery](https://mlsanity.com/paper/2606.19624/) (2026)
- [Beyond Simple Concatenation: Fairly Assessing PLM Architectures for Multi-Chain Protein-Protein Interactions Prediction](https://mlsanity.com/paper/2505.20036/) (2025)
- [PepSpecBench: A Unified Evaluation Benchmark for Peptide Tandem Mass Spectrometry Prediction](https://mlsanity.com/paper/2605.01945/) (2026)
- [Enhancing Protein-Protein Interaction Prediction with Hierarchical Motif-based Multimodal Protein Embedding](https://mlsanity.com/paper/2606.02629/) (2026)
- [ReLaG: A Scalable Framework Generalizing Random Splits to Data with Latent Relations](https://mlsanity.com/paper/2609.38538/) (2026)

## Abstract (arXiv)

In recent years, there has been remarkable progress in machine learning for protein-protein interactions. However, prior work has predominantly focused on improving learning algorithms, with less attention paid to evaluation strategies and data preparation. Here, we demonstrate that further development of machine learning methods may be hindered by the quality of existing train-test splits. Specifically, we find that commonly used splitting strategies for protein complexes, based on protein sequence or metadata similarity, introduce major data leakage. This may result in overoptimistic evaluation of generalization, as well as unfair benchmarking of the models, biased towards assessing their overfitting capacity rather than practical utility. To overcome the data leakage, we recommend constructing data splits based on 3D structural similarity of protein-protein interfaces and suggest corresponding algorithms. We believe that addressing the data leakage problem is critical for further progress in this research area.