# Contrastive Pretraining for Echocardiography Segmentation with Limited Data

arXiv: https://arxiv.org/abs/2201.07219 · HTML: https://mlsanity.com/paper/2201.07219/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 18 references that are in the map.
- First posted on arXiv 16 Jan 2022.

## Builds on (most cited first, 10 of 10)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs](https://mlsanity.com/paper/1606.00915/) (2016)
- [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1802.02611/) (2018)
- [Rethinking Atrous Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1706.05587/) (2017)
- [Deep Learning for Segmentation using an Open Large-Scale Dataset in 2D Echocardiography](https://mlsanity.com/paper/1908.06948/) (2019)
- [Adaptivity without Compromise: A Momentumized, Adaptive, Dual Averaged Gradient Method for Stochastic Optimization](https://mlsanity.com/paper/2101.11075/) (2021)
- [Contrastive Learning for View Classification of Echocardiograms](https://mlsanity.com/paper/2108.03124/) (2021)

## Built on by (most cited first, 13 of 13)

- [EchoCoTr: Estimation of the Left Ventricular Ejection Fraction from Spatiotemporal Echocardiography](https://mlsanity.com/paper/2209.04242/) (2022)
- [Label-free segmentation from cardiac ultrasound using self-supervised learning](https://mlsanity.com/paper/2210.04979/) (2022)
- [Self-supervised contrastive learning of echocardiogram videos enables label-efficient cardiac disease diagnosis](https://mlsanity.com/paper/2207.11581/) (2022)
- [Self-Supervised Pretraining Improves Performance and Inference Efficiency in Multiple Lung Ultrasound Interpretation Tasks](https://mlsanity.com/paper/2309.02596/) (2023)
- [Exploring the Utility of Self-Supervised Pretraining Strategies for the Detection of Absent Lung Sliding in M-Mode Lung Ultrasound](https://mlsanity.com/paper/2304.02724/) (2023)
- [A Survey of the Impact of Self-Supervised Pretraining for Diagnostic Tasks with Radiological Images](https://mlsanity.com/paper/2309.02555/) (2023)
- [Real-time Automatic M-mode Echocardiography Measurement with Panel Attention from Local-to-Global Pixels](https://mlsanity.com/paper/2308.07717/) (2023)
- [SimLVSeg: Simplifying Left Ventricular Segmentation in 2D+Time Echocardiograms with Self- and Weakly-Supervised Learning](https://mlsanity.com/paper/2310.00454/) (2023)
- [A texture-based framework for foundational ultrasound models](https://mlsanity.com/paper/2602.01444/) (2026)
- [EAGT: Echocardiography Augmentation for Generalisability and Transferability](https://mlsanity.com/paper/2605.16427/) (2026)
- [Unlocking the Heart Using Adaptive Locked Agnostic Networks](https://mlsanity.com/paper/2309.11899/) (2023)
- [Leveraging Self-Supervised Learning for Fetal Cardiac Planes Classification using Ultrasound Scan Videos](https://mlsanity.com/paper/2407.21738/) (2024)
- [The Efficacy of Semantics-Preserving Transformations in Self-Supervised Learning for Medical Ultrasound](https://mlsanity.com/paper/2504.07904/) (2025)

## Abstract (arXiv)

Contrastive learning has proven useful in many applications where access to labelled data is limited. The lack of annotated data is particularly problematic in medical image segmentation as it is difficult to have clinical experts manually annotate large volumes of data such as cardiac structures in ultrasound images of the heart. In this paper, We propose a self supervised contrastive learning method to segment the left ventricle from echocardiography when limited annotated images exist. Furthermore, we study the effect of contrastive pretraining on two well-known segmentation networks, UNet and DeepLabV3. Our results show that contrastive pretraining helps improve the performance on left ventricle segmentation, particularly when annotated data is scarce. We show how to achieve comparable results to state-of-the-art fully supervised algorithms when we train our models in a self-supervised fashion followed by fine-tuning on just 5% of the data. We show that our solution outperforms what is currently published on a large public dataset (EchoNet-Dynamic) achieving a Dice score of 0.9252. We also compare the performance of our solution on another smaller dataset (CAMUS) to demonstrate the generalizability of our proposed solution. The code is available at (https://github.com/BioMedIA-MBZUAI/contrastive-echo).