# Contrastive Learning under Noisy Temporal Self-Supervision for Colonoscopy Videos

arXiv: https://arxiv.org/abs/2605.12320 · HTML: https://mlsanity.com/paper/2605.12320/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 21 of its 34 references that are in the map.
- First posted on arXiv 12 May 2026.

## Builds on (most cited first, 21 of 21)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Supervised Contrastive Learning](https://mlsanity.com/paper/2004.11362/) (2020)
- [DINOv3](https://mlsanity.com/paper/2508.10104/) (2025)
- [V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning](https://mlsanity.com/paper/2506.09985/) (2025)
- [End-to-End Learning of Visual Representations from Uncurated Instructional Videos](https://mlsanity.com/paper/1912.06430/) (2019)
- [Video Polyp Segmentation: A Deep Learning Perspective](https://mlsanity.com/paper/2203.14291/) (2022)
- [Foundation Model for Endoscopy Video Analysis via Large-scale Self-supervised Pre-train](https://mlsanity.com/paper/2306.16741/) (2023)
- [Unbiased Supervised Contrastive Learning](https://mlsanity.com/paper/2211.05568/) (2022)
- [Colonoscopy Polyp Detection and Classification: Dataset Creation and Comparative Evaluations](https://mlsanity.com/paper/2104.10824/) (2021)
- [Scaling up self-supervised learning for improved surgical foundation models](https://mlsanity.com/paper/2501.09436/) (2025)
- [REAL-Colon: A dataset for developing real-world AI applications in colonoscopy](https://mlsanity.com/paper/2403.02163/) (2024)
- [Integrating Prior Knowledge in Contrastive Learning with Kernel](https://mlsanity.com/paper/2206.01646/) (2022)
- [Self-Supervised Polyp Re-Identification in Colonoscopy](https://mlsanity.com/paper/2306.08591/) (2023)
- [Towards Polyp Counting In Full-Procedure Colonoscopy Videos](https://mlsanity.com/paper/2502.10054/) (2025)
- [Temporally-Aware Supervised Contrastive Learning for Polyp Counting in Colonoscopy](https://mlsanity.com/paper/2507.02493/) (2025)

## Abstract (arXiv)

Learning robust representations of polyp tracklets is key to enabling multiple AI-assisted colonoscopy applications, from polyp characterization to automated reporting and retrieval. Supervised contrastive learning is an effective approach for learning such representations, but it typically relies on correct positive and negative definitions. Collecting these labels requires linking tracklets that depict the same underlying polyp entity throughout the video, which is costly and demands specialized clinical expertise. In this work, we leverage the sequential workflow of colonoscopy procedures to derive self-supervised associations from temporal structure. Since temporally derived associations are not guaranteed to be correct, we introduce a noise-aware contrastive loss to account for noisy associations. We demonstrate the effectiveness of the learned representations across multiple downstream tasks, including polyp retrieval and re-identification, size estimation, and histology classification. Our method outperforms prior self-supervised and supervised baselines, and matches or exceeds recent foundation models across all tasks, using a lightweight encoder trained on only 27 videos. Code is available at https://github.com/lparolari/ntssl.