# Colo-SCRL: Self-Supervised Contrastive Representation Learning for Colonoscopic Video Retrieval

arXiv: https://arxiv.org/abs/2303.15671 · HTML: https://mlsanity.com/paper/2303.15671/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 27 references that are in the map.
- First posted on arXiv 28 Mar 2023.

## Builds on (most cited first, 16 of 16)

- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Unsupervised Feature Learning via Non-Parametric Instance-level Discrimination](https://mlsanity.com/paper/1805.01978/) (2018)
- [VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Training](https://mlsanity.com/paper/2203.12602/) (2022)
- [Kvasir-SEG: A Segmented Polyp Dataset](https://mlsanity.com/paper/1911.07069/) (2019)
- [Spatiotemporal Contrastive Video Representation Learning](https://mlsanity.com/paper/2008.03800/) (2020)
- [Self-supervised Co-training for Video Representation Learning](https://mlsanity.com/paper/2010.09709/) (2020)
- [VideoMoCo: Contrastive Video Representation Learning with Temporally Adversarial Examples](https://mlsanity.com/paper/2103.05905/) (2021)
- [EndoMapper dataset of complete calibrated endoscopy procedures](https://mlsanity.com/paper/2204.14240/) (2022)
- [Heidelberg Colorectal Data Set for Surgical Data Science in the Sensor Operating Room](https://mlsanity.com/paper/2005.03501/) (2020)
- [ViSiL: Fine-grained Spatio-Temporal Video Similarity Learning](https://mlsanity.com/paper/1908.07410/) (2019)
- [Self-supervised Video Representation Learning by Uncovering Spatio-temporal Statistics](https://mlsanity.com/paper/2008.13426/) (2020)
- [Less is More: Learning from Synthetic Data with Fine-grained Attributes for Person Re-Identification](https://mlsanity.com/paper/2109.10498/) (2021)
- [DnS: Distill-and-Select for Efficient and Accurate Video Indexing and Retrieval](https://mlsanity.com/paper/2106.13266/) (2021)
- [Motion-based Camera Localization System in Colonoscopy Videos](https://mlsanity.com/paper/2012.01690/) (2020)

## Built on by (most cited first, 6 of 6)

- [Know Your Self-supervised Learning: A Survey on Image-based Generative and Discriminative Training](https://mlsanity.com/paper/2305.13689/) (2023)
- [Learning Multi-axis Representation in Frequency Domain for Medical Image Segmentation](https://mlsanity.com/paper/2312.17030/) (2023)
- [Learning Collaborative Knowledge with Multimodal Representation for Polyp Re-Identification](https://mlsanity.com/paper/2408.05914/) (2024)
- [Temporally-Aware Supervised Contrastive Learning for Polyp Counting in Colonoscopy](https://mlsanity.com/paper/2507.02493/) (2025)
- [Learning Discriminative Visual-Text Representation for Polyp Re-Identification](https://mlsanity.com/paper/2307.10625/) (2023)
- [GPF-Net: Gated Progressive Fusion Learning for Polyp Re-Identification](https://mlsanity.com/paper/2512.21476/) (2025)

## Abstract (arXiv)

Colonoscopic video retrieval, which is a critical part of polyp treatment, has great clinical significance for the prevention and treatment of colorectal cancer. However, retrieval models trained on action recognition datasets usually produce unsatisfactory retrieval results on colonoscopic datasets due to the large domain gap between them. To seek a solution to this problem, we construct a large-scale colonoscopic dataset named Colo-Pair for medical practice. Based on this dataset, a simple yet effective training method called Colo-SCRL is proposed for more robust representation learning. It aims to refine general knowledge from colonoscopies through masked autoencoder-based reconstruction and momentum contrast to improve retrieval performance. To the best of our knowledge, this is the first attempt to employ the contrastive learning paradigm for medical video retrieval. Empirical results show that our method significantly outperforms current state-of-the-art methods in the colonoscopic video retrieval task.