mlsanityData to 14 Oct 2026

ModAn-MulSupCon: Modality-and Anatomy-Aware Multi-Label Supervised Contrastive Pretraining for Medical Imaging

arXiv 2508.18613 ↗First posted 26 Aug 2025eess.IV cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Background and objective: Expert annotations limit large-scale supervised pretraining in medical imaging, while ubiquitous metadata (modality, anatomical region) remain underused. We introduce ModAn-MulSupCon, a modality- and anatomy-aware multi-label supervised contrastive pretraining method that leverages such metadata to learn transferable representations. Method: Each image's modality and anatomy are encoded as a multi-hot vector. A ResNet-18 encoder is pretrained on a mini subset of RadImageNet (miniRIN, 16,222 images) with a Jaccard-weighted multi-label supervised contrastive loss, and then evaluated by fine-tuning and linear probing on three binary classification tasks--ACL tear (knee MRI), lesion malignancy (breast ultrasound), and nodule malignancy (thyroid ultrasound). Result: With fine-tuning, ModAn-MulSupCon achieved the best AUC on MRNet-ACL (0.964) and Thyroid (0.763), surpassing all baselines (p<0.05), and ranked second on Breast (0.926) behind SimCLR (0.940; not significant). With the encoder frozen, SimCLR/ImageNet were superior, indicating that ModAn-MulSupCon representations benefit most from task adaptation rather than linear separability. Conclusion: Encoding readily available modality/anatomy metadata as multi-label targets provides a practical, scalable pretraining signal that improves downstream accuracy when fine-tuning is feasible. ModAn-MulSupCon is a strong initialization for label-scarce clinical settings, whereas SimCLR/ImageNet remain preferable for frozen-encoder deployments.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 23 of its 45 references that are in the map. First posted on arXiv 26 Aug 2025.

Builds on

23 of 45 references.

References · 23

  1. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  2. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  3. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  4. 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,898
  5. 2020Supervised Contrastive Learningcited by 2,829
  6. 2019CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparisoncited by 1,423
  7. 2021Self-Supervised Pre-Training of Swin Transformers for 3D Medical Image Analysiscited by 422
  8. 2022Making the Most of Text Semantics to Improve Biomedical Vision--Language Processingcited by 339
  9. 2019Multi-Label Image Recognition with Graph Convolutional Networkscited by 322
  10. 2016CNN-RNN: A Unified Framework for Multi-label Image Classificationcited by 190
  11. 2018AttentionXML: Label Tree-based Attention-Aware Deep Model for High-Performance Extreme Multi-Label Text Classificationcited by 102
  12. 2017Learning Spatial Regularization with Image-level Supervisions for Multi-label Image Classificationcited by 85
11 more
  1. 2022Self Pre-training with Masked Autoencoders for Medical Image Classification and Segmentationcited by 69
  2. 2022Use All The Labels: A Hierarchical Multi-Label Contrastive Learning Frameworkcited by 59
  3. 2019ImageGCN: Multi-Relational Image Graph Convolutional Networks for Disease Identification with Chest X-rayscited by 22
  4. 2023Advancing Volumetric Medical Image Segmentation via Global-Local Masked Autoencodercited by 21
  5. 2024MiM: Mask in Mask Self-Supervised Pre-Training for 3D Medical Image Analysiscited by 19
  6. 2022Swin MAE: Masked Autoencoders for Small Datasetscited by 17
  7. 2022Distributed Contrastive Learning for Medical Image Segmentationcited by 16
  8. 2021SCALP -- Supervised Contrastive Learning for Cardiopulmonary Disease Classification and Localization in Chest X-rays using Patient Metadatacited by 9
  9. 2024HEX: Hierarchical Emergence Exploitation in Self-Supervised Algorithmscited by 7
  10. 2025Hypergraph Tversky-Aware Domain Incremental Learning for Brain Tumor Segmentation with Missing Modalitiescited by 7
  11. 2024VIS-MAE: An Efficient Self-supervised Learning Approach on Medical Image Segmentation and Classificationcited by 4

Built on by

Not cited yet.