mlsanityData to 14 Oct 2026

Which Pretext Task Transfers? Self-Supervised Pretraining Objectives for Lung Ultrasound

arXiv 2609.16551 ↗First posted 15 Sep 2026cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Self-supervised learning (SSL) can reduce the need for labelled medical images, but the choice of pretext objective remains unclear for lung ultrasound (LUS). Contrastive learning, masked reconstruction, and joint-embedding predictive architectures (JEPA) differ in the space in which their targets are defined, yet existing ultrasound studies compare them under different corpora, backbones, and evaluation protocols. We compare these three objective families using the same encoder backbone, pretraining corpus, optimisation schedule, and frozen-evaluation protocol. Encoders are pretrained on COVID-BLUeS LUS videos and evaluated with linear, kNN, and attentive probes at 5%, 10%, 50%, and 100% label budgets. Evaluation is performed on POCUS using patient-level five-fold cross-validation and on the independently acquired Mendeley-Uganda dataset, which is excluded from both pretraining and probe fitting. At the full label budget under linear probing, VideoMAE and V-JEPA achieve 66.5 ± 13.1 and 65.4 ± 11.7 balanced accuracy on POCUS, while MoCo achieves 42.1 ± 1.2. On Mendeley-Uganda, the ranking reverses: MoCo performs best at 62.7 ± 1.0, followed by VideoMAE at 53.8 ± 2.8, while V-JEPA falls near chance at 35.1 ± 4.9. These results show that POCUS probe accuracy alone does not identify the objective that transfers best across datasets. We also outline planned representation-level analyses to examine this reversal. Code is publicly available at https://github.com/moeinheidari7829/LUSVideoSSL.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 19 of its 25 references that are in the map. First posted on arXiv 15 Sep 2026.

Builds on

19 of 25 references.

References · 19

  1. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  2. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  3. 2021An Empirical Study of Training Self-Supervised Vision Transformerscited by 1,616
  4. 2025DINOv3cited by 1,414
  5. 2022VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Trainingcited by 1,322
  6. 2023Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecturecited by 1,039
  7. 2025V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planningcited by 809
  8. 2021VideoMoCo: Contrastive Video Representation Learning with Temporally Adversarial Examplescited by 154
  9. 2023USFM: A Universal Ultrasound Foundation Model Generalized to Tasks and Organs towards Label Efficient Image Analysiscited by 68
  10. 2024Learning by Reconstruction Produces Uninformative Features For Perceptioncited by 55
  11. 2025Joint Embedding vs Reconstruction: Provable Benefits of Latent Space Prediction for Self Supervised Learningcited by 36
  12. 2020USCL: Pretraining Deep Ultrasound Image Diagnosis Model through Video Contrastive Representation Learningcited by 34
7 more
  1. 2024How JEPA Avoids Noisy Features: The Implicit Bias of Deep Linear Self Distillation Networkscited by 31
  2. 2025USF-MAE: Ultrasound Self-Supervised Foundation Model with Masked Autoencodingcited by 16
  3. 2025COVID-BLUeS -- A Prospective Study on the Value of AI in Lung Ultrasound Analysiscited by 7
  4. 2021Tasting the cake: evaluating self-supervised generalization on out-of-distribution multimodal MRI datacited by 5
  5. 2025Self-supervised Learning of Echocardiographic Video Representations via Online Cluster Distillationcited by 4
  6. 2025Self-Supervised Ultrasound-Video Segmentation with Feature Prediction and 3D Localised Losscited by 4
  7. 2026Pretext Matters: An Empirical Study of SSL Methods in Medical Imagingcited by 3

Built on by

Not cited yet.