# CCPL: Cross-modal Contrastive Protein Learning

arXiv: https://arxiv.org/abs/2303.11783 · HTML: https://mlsanity.com/paper/2303.11783/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 28 references that are in the map.
- First posted on arXiv 19 Mar 2023.

## Builds on (most cited first, 9 of 9)

- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Learning from Protein Structure with Geometric Vector Perceptrons](https://mlsanity.com/paper/2009.01411/) (2020)
- [Evaluating Protein Transfer Learning with TAPE](https://mlsanity.com/paper/1906.08230/) (2019)
- [Protein Representation Learning by Geometric Structure Pretraining](https://mlsanity.com/paper/2203.06125/) (2022)
- [CVT-SLR: Contrastive Visual-Textual Transformation for Sign Language Recognition with Variational Alignment](https://mlsanity.com/paper/2303.05725/) (2023)
- [AlphaDesign: A graph protein design method and benchmark on AlphaFoldDB](https://mlsanity.com/paper/2202.01079/) (2022)
- [Contrastive Representation Learning for 3D Protein Structures](https://mlsanity.com/paper/2205.15675/) (2022)
- [Using Context-to-Vector with Graph Retrofitting to Improve Word Embeddings](https://mlsanity.com/paper/2210.16848/) (2022)
- [Leveraging Graph-based Cross-modal Information Fusion for Neural Sign Language Translation](https://mlsanity.com/paper/2211.00526/) (2022)

## Built on by (most cited first, 4 of 4)

- [Protein Large Language Models: A Comprehensive Survey](https://mlsanity.com/paper/2502.17504/) (2025)
- [Protein 3D Graph Structure Learning for Robust Structure-based Protein Property Prediction](https://mlsanity.com/paper/2310.11466/) (2023)
- [Cross-Gate MLP with Protein Complex Invariant Embedding is A One-Shot Antibody Designer](https://mlsanity.com/paper/2305.09480/) (2023)
- [Structure-Aligned Protein Language Model](https://mlsanity.com/paper/2505.16896/) (2025)

## Abstract (arXiv)

Effective protein representation learning is crucial for predicting protein functions. Traditional methods often pretrain protein language models on large, unlabeled amino acid sequences, followed by finetuning on labeled data. While effective, these methods underutilize the potential of protein structures, which are vital for function determination. Common structural representation techniques rely heavily on annotated data, limiting their generalizability. Moreover, structural pretraining methods, similar to natural language pretraining, can distort actual protein structures. In this work, we introduce a novel unsupervised protein structure representation pretraining method, cross-modal contrastive protein learning (CCPL). CCPL leverages a robust protein language model and uses unsupervised contrastive alignment to enhance structure learning, incorporating self-supervised structural constraints to maintain intrinsic structural information. We evaluated our model across various benchmarks, demonstrating the framework's superiority.