mlsanityData to 7 Oct 2026

Mammo-CLIP: A Vision Language Foundation Model to Enhance Data Efficiency and Robustness in Mammography

arXiv 2405.12255 ↗First posted 20 May 2024eess.IV cs.CVcited by 35

On the map: Multimodal LLMs / vision-language models → Medical vision-language models

Abstract (arXiv)

The lack of large and diverse training data on Computer-Aided Diagnosis (CAD) in breast cancer detection has been one of the concerns that impedes the adoption of the system. Recently, pre-training with large-scale image text datasets via Vision-Language models (VLM) (eg CLIP) partially addresses the issue of robustness and data efficiency in computer vision (CV). This paper proposes Mammo-CLIP, the first VLM pre-trained on a substantial amount of screening mammogram-report pairs, addressing the challenges of dataset diversity and size. Our experiments on two public datasets demonstrate strong performance in classifying and localizing various mammographic attributes crucial for breast cancer detection, showcasing data efficiency and robustness similar to CLIP in CV. We also propose Mammo-FActOR, a novel feature attribution method, to provide spatial interpretation of representation with sentence-level granularity within mammography reports. Code is available publicly: urlhttps://github.com/batmanlab/Mammo-CLIP.

Key facts. Cited by 35 papers in the map (data through 7 Oct 2026). Placed in Medical vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 18 of its 27 references that are in the map. First posted on arXiv 20 May 2024.

Builds on

18 of 27 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023CXR-CLIP: Toward Large Scale Chest X-ray Language-Image Pre-training20 Oct

Other references · 17

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  3. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  4. 2019Publicly Available Clinical BERT Embeddingscited by 995
  5. 2020Contrastive Learning of Medical Visual Representations from Paired Images and Textcited by 636
  6. 2022MedCLIP: Contrastive Learning from Unpaired Medical Images and Textcited by 564
  7. 2021Supervision Exists Everywhere: A Data Efficient Contrastive Language-Image Pre-training Paradigmcited by 405
  8. 2022Language in a Bottle: Language Model Guided Concept Bottlenecks for Interpretable Image Classificationcited by 310
  9. 2023Label-Free Concept Bottleneck Modelscited by 282
  10. 2023CLIP-Driven Universal Model for Organ Segmentation and Tumor Detectioncited by 224
  11. 2023PMC-CLIP: Contrastive Language-Image Pre-training using Biomedical Documentscited by 212
  12. 2023MedKLIP: Medical Knowledge Enhanced Language-Image Pre-Training in Radiologycited by 149
5 more
  1. 2021Does CLIP Benefit Visual Question Answering in the Medical Domain as Much as it Does in the General Domain?cited by 85
  2. 2022VinDr-Mammo: A large-scale benchmark dataset for computer-aided diagnosis in full-field digital mammographycited by 83
  3. 2022Anatomy-Guided Weakly-Supervised Abnormality Localization in Chest X-rayscited by 21
  4. 2023Anatomy-Driven Pathology Detection on Chest X-rayscited by 10
  5. 2023ViLLA: Fine-Grained Vision-Language Representation Learning from Real-World Datacited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 4; 2025: 14; 2026: 17.

35 papers.

Sort:

By area:

  1. 2024A Comprehensive Survey of Scientific Large Language Models and Their Applications in Scientific Discoverycited by 96
  2. 2024Robust image representations with counterfactual contrastive learningcited by 12
  3. 2025Enhancing SAM with Efficient Prompting and Preference Optimization for Semi-supervised Medical Image Segmentationcited by 12
  4. 2024Multi-View and Multi-Scale Alignment for Contrastive Language-Image Pre-training in Mammographycited by 9
  5. 2024LADDER: Language-Driven Slice Discovery and Error Rectification in Vision Classifierscited by 7
  6. 2025Bias and Generalizability of Foundation Models across Datasets in Breast Mammographycited by 3
  7. 2025Mammo-SAE: Interpreting Breast Cancer Concept Learning with Sparse Autoencoderscited by 2
  8. 2025Exemplar Med-DETR: Toward Generalized and Robust Lesion Detection in Mammogram Images and beyondcited by 2
  9. 2025Position: Restructuring of Categories and Implementation of Guidelines Essential for VLM Adoption in Healthcarecited by 1
  10. 2025Enhancing the Safety of Medical Vision-Language Models by Synthetic Demonstrationscited by 1
  11. 2025Comparison of ConvNeXt and Vision-Language Models for Breast Density Assessment in Screening Mammographycited by 1
  12. 2025Data-Efficient Fine-Tuning of Vision-Language Models for Diagnosis of Alzheimer's Diseasecited by 1
23 more
  1. 2025GLAM: Geometry-Guided Local Alignment for Multi-View VLP in Mammographycited by 1
  2. 2025MammoClean: Toward Reproducible and Bias-Aware AI in Mammography through Dataset Harmonizationcited by 1
  3. 2026MIL-PF: Multiple Instance Learning on Precomputed Features for Mammography Classificationcited by 1
  4. 2026Attend what matters: Leveraging vision foundational models for breast cancer classification using mammogramscited by 1
  5. 2026Dataset-Origin Signatures and Shortcut Learning in Screening Mammography AI: A Cross-Dataset Case Studycited by 1
  6. 2025MAMBO: High-Resolution Generative Approach for Mammography Imagescited by 0
  7. 2025Mammo-CLIP Dissect: A Framework for Analysing Mammography Concepts in Vision-Language Modelscited by 0
  8. 2025Breast Cancer VLMs: Clinically Practical Vision-Language Train-Inference Modelscited by 0
  9. 2025MV-MLM: Bridging Multi-View Mammography and Language for Breast Cancer Diagnosis and Risk Predictioncited by 0
  10. 2026Does medical specialization of VLMs enhance discriminative power?: A comprehensive investigation through feature distribution analysiscited by 0
  11. 2026Free Lunch in Medical Image Foundation Model Pre-training via Randomized Synthesis and Disentanglementcited by 0
  12. 2026MammoWise: Multi-Model Local RAG Pipeline for Mammography Report Generationcited by 0
  13. 2026OTCHA: Optimal Transport-driven Confidence-aware Latent Hub Alignment for Multi-View Medical Image Classificationcited by 0
  14. 2026MammoFlow: Multiview Mammogram Synthesis with Anatomically Consistent Flow Matchingcited by 0
  15. 2026CPR: Chained Perceptual Refinement for Coarse-to-Fine Medical Image Classificationcited by 0
  16. 2026Benchmarking the Robustness of Foundation Models for Mammography under Domain Shiftcited by 0
  17. 2026Pitfalls of Administrative Censoring in Survival Models with Time-Indexed Inputscited by 0
  18. 2026Region-Grounded Vision-Language Learning for Detection-Guided Mammographic Lesion Classificationcited by 0
  19. 2026Less Annotation, More Interpretation: Prior-Guided Concept Bottleneck Models for Interpretable Cancer Imaging Diagnosiscited by 0
  20. 2026TRUST: Threshold-Recalibrated Uncertainty-Safe Training for Certified Dismissal in Breast Cancer Screeningcited by 0
  21. 2026Solving the Needle-in-a-Haystack Problem in Mammography Vision-Language Model with Differentiable Subset Samplingcited by 0
  22. 2026Mammography Foundation Models for Opportunistic Prediction of Major Adverse Cardiovascular Eventscited by 0
  23. 2026Mammo-LIFE: Longitudinal Mammographic Imaging and Clinical Feature Enrichment for Post-Radiotherapy Outcome Predictioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Mammo-CLIP: Leveraging Contrastive Language-Image Pre-training (CLIP) for Enhanced Breast Cancer Diagnosis with Multi-view Mammography
  2. 2026MAM-CLIP: Vision-Language Pretraining on Mammography Atlases for BI-RADS Classification
  3. 2024Knowledge-grounded Adaptation Strategy for Vision-language Models: Building Unique Case-set for Screening Mammograms for Residents Training
  4. 2026MammoMix: Leveraging Mixture of Experts for Robust Mammogram Breast Detection
  5. 2025AMRG: Extend Vision Language Models for Automatic Mammography Report Generation
  6. 2024On Large Visual Language Models for Medical Imaging Analysis: An Empirical Study
  7. 2026From Density to Biopsy Decisions and Malignancy Prediction: A Benchmark Study of Multimodal Large Language Models Against Radiologists in Digital and Contrast-Enhanced Mammography
  8. 2025Mammo-Mamba: A Hybrid State-Space and Transformer Architecture with Sequential Mixture of Experts for Multi-View Mammography