# Unsupervised Contrastive Learning of Image Representations from Ultrasound Videos with Hard Negative Mining

arXiv: https://arxiv.org/abs/2207.13148 · HTML: https://mlsanity.com/paper/2207.13148/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 28 references that are in the map.
- First posted on arXiv 26 Jul 2022.

## Builds on (most cited first, 12 of 12)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [Exploring Simple Siamese Representation Learning](https://mlsanity.com/paper/2011.10566/) (2020)
- [Unsupervised Representation Learning by Predicting Image Rotations](https://mlsanity.com/paper/1803.07728/) (2018)
- [COVID-CT-Dataset: A CT Scan Dataset about COVID-19](https://mlsanity.com/paper/2003.13865/) (2020)
- [USCL: Pretraining Deep Ultrasound Image Diagnosis Model through Video Contrastive Representation Learning](https://mlsanity.com/paper/2011.13066/) (2020)
- [POCOVID-Net: Automatic Detection of COVID-19 From a New Lung Ultrasound Imaging Dataset (POCUS)](https://mlsanity.com/paper/2004.12084/) (2020)
- [Contrastive Learning of Image Representations with Cross-Video Cycle-Consistency](https://mlsanity.com/paper/2105.06463/) (2021)
- [Surpassing the Human Accuracy: Detecting Gallbladder Cancer from USG Images with Curriculum Learning](https://mlsanity.com/paper/2204.11433/) (2022)

## Built on by (most cited first, 14 of 14)

- [FocusMAE: Gallbladder Cancer Detection from Ultrasound Videos with Focused Masked Autoencoders](https://mlsanity.com/paper/2403.08848/) (2024)
- [Procedure Learning via Regularized Gromov-Wasserstein Optimal Transport](https://mlsanity.com/paper/2507.15540/) (2025)
- [Self-Supervised Pretraining Improves Performance and Inference Efficiency in Multiple Lung Ultrasound Interpretation Tasks](https://mlsanity.com/paper/2309.02596/) (2023)
- [Gall Bladder Cancer Detection from US Images with Only Image Level Labels](https://mlsanity.com/paper/2309.05261/) (2023)
- [Focus on Texture: Rethinking Pre-training in Masked Autoencoders for Medical Image Classification](https://mlsanity.com/paper/2507.10869/) (2025)
- [Exploring the Utility of Self-Supervised Pretraining Strategies for the Detection of Absent Lung Sliding in M-Mode Lung Ultrasound](https://mlsanity.com/paper/2304.02724/) (2023)
- [A Survey of the Impact of Self-Supervised Pretraining for Diagnostic Tasks with Radiological Images](https://mlsanity.com/paper/2309.02555/) (2023)
- [Thyroid ultrasound diagnosis improvement via multi-view self-supervised learning and two-stage pre-training](https://mlsanity.com/paper/2402.11497/) (2024)
- [Intra-video Positive Pairs in Self-Supervised Learning for Ultrasound](https://mlsanity.com/paper/2403.07715/) (2024)
- [Anatomy-Anchored Self-Supervision: Distilling Vision Foundation Models for Invariant Ultrasound Representation](https://mlsanity.com/paper/2605.25402/) (2026)
- [LQ-Adapter: ViT-Adapter with Learnable Queries for Gallbladder Cancer Detection from Ultrasound Image](https://mlsanity.com/paper/2412.00374/) (2024)
- [The Efficacy of Semantics-Preserving Transformations in Self-Supervised Learning for Medical Ultrasound](https://mlsanity.com/paper/2504.07904/) (2025)
- [Assessing Risk of Stealing Proprietary Models for Medical Imaging Tasks](https://mlsanity.com/paper/2506.19464/) (2025)
- [UltrasODM: A Dual Stream Optical Flow Mamba Network for 3D Freehand Ultrasound Reconstruction](https://mlsanity.com/paper/2512.07756/) (2025)

## Abstract (arXiv)

Rich temporal information and variations in viewpoints make video data an attractive choice for learning image representations using unsupervised contrastive learning (UCL) techniques. State-of-the-art (SOTA) contrastive learning techniques consider frames within a video as positives in the embedding space, whereas the frames from other videos are considered negatives. We observe that unlike multiple views of an object in natural scene videos, an Ultrasound (US) video captures different 2D slices of an organ. Hence, there is almost no similarity between the temporally distant frames of even the same US video. In this paper we propose to instead utilize such frames as hard negatives. We advocate mining both intra-video and cross-video negatives in a hardness-sensitive negative mining curriculum in a UCL framework to learn rich image representations. We deploy our framework to learn the representations of Gallbladder (GB) malignancy from US videos. We also construct the first large-scale US video dataset containing 64 videos and 15,800 frames for learning GB representations. We show that the standard ResNet50 backbone trained with our framework improves the accuracy of models pretrained with SOTA UCL techniques as well as supervised pretrained models on ImageNet for the GB malignancy detection task by 2-6%. We further validate the generalizability of our method on a publicly available lung US image dataset of COVID-19 pathologies and show an improvement of 1.5% compared to SOTA. Source code, dataset, and models are available at https://gbc-iitd.github.io/usucl.