mlsanityData to 7 Oct 2026

CXR-CLIP: Toward Large Scale Chest X-ray Language-Image Pre-training

arXiv 2310.13292 ↗First posted 20 Oct 2023cs.CV cs.LGcited by 81

On the map: Multimodal LLMs / vision-language models → Medical vision-language models

Abstract (arXiv)

A large-scale image-text pair dataset has greatly contributed to the development of vision-language pre-training (VLP) models, which enable zero-shot or few-shot classification without costly annotation. However, in the medical domain, the scarcity of data remains a significant challenge for developing a powerful VLP model. In this paper, we tackle the lack of image-text data in chest X-ray by expanding image-label pair as image-text pair via general prompt and utilizing multiple images and multiple sections in a radiologic report. We also design two contrastive losses, named ICL and TCL, for learning study-level characteristics of medical images and reports, respectively. Our model outperforms the state-of-the-art models trained under the same conditions. Also, enlarged dataset improve the discriminative power of our pre-trained model for classification, while sacrificing marginal retrieval performance. Code is available at https://github.com/kakaobrain/cxr-clip.

Key facts. Cited by 81 papers in the map (data through 7 Oct 2026). Placed in Medical vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 20 of its 30 references that are in the map. First posted on arXiv 20 Oct 2023.

Builds on

20 of 30 references.

References · 20

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  5. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  6. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
  7. 2017Decoupled Weight Decay Regularizationcited by 1,722
  8. 2019CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparisoncited by 1,410
  9. 2019Publicly Available Clinical BERT Embeddingscited by 995
  10. 2017ChestX-ray8: Hospital-scale Chest X-ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseasescited by 969
  11. 2019EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Taskscited by 800
  12. 2021SLIP: Self-supervision meets Language-Image Pre-trainingcited by 434
8 more
  1. 2021Supervision Exists Everywhere: A Data Efficient Contrastive Language-Image Pre-training Paradigmcited by 405
  2. 2022Making the Most of Text Semantics to Improve Biomedical Vision--Language Processingcited by 336
  3. 2020VinDr-CXR: An open dataset of chest X-rays with radiologist's annotationscited by 236
  4. 2022Unified Contrastive Learning in Image-Text-Label Spacecited by 194
  5. 2021Generalized Radiograph Representation Learning via Cross-supervision between Images and Free-text Radiology Reportscited by 80
  6. 2023Advancing Radiograph Representation Learning with Masked Record Modelingcited by 75
  7. 2021MedAug: Contrastive learning leveraging patient metadata improves representations for chest X-ray interpretationcited by 52
  8. 2022Significantly improving zero-shot X-ray pathology classification via fine-tuning pre-trained image-text encoderscited by 15

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 28; 2025: 31; 2026: 19.

81 papers.

Sort:

By area:

  1. 2024A Comprehensive Survey of Scientific Large Language Models and Their Applications in Scientific Discoverycited by 96
  2. 2024Foundation Model for Advancing Healthcare: Challenges, Opportunities, and Future Directionscited by 77
  3. 2024FairCLIP: Harnessing Fairness in Vision-Language Learningcited by 69
  4. 2024A Comprehensive Survey of Foundation Models in Medicinecited by 56
  5. 2023CLIP in Medical Imaging: A Surveycited by 40
  6. 2024Mammo-CLIP: A Vision Language Foundation Model to Enhance Data Efficiency and Robustness in Mammographycited by 35
  7. 2024CXPMRG-Bench: Pre-training and Benchmarking for X-ray Medical Report Generation on CheXpert Plus Datasetcited by 33
  8. 2024CT-GLIP: 3D Grounded Language-Image Pretraining with CT Scans and Radiology Reports for Full-Body Scenarioscited by 29
  9. 2024GEMeX: A Large-Scale, Groundable, and Explainable Medical VQA Benchmark for Chest X-ray Diagnosiscited by 28
  10. 2024Test-Time Adaptation with SaLIP: A Cascade of SAM and CLIP for Zero shot Medical Image Segmentationcited by 25
  11. 2024ECG Semantic Integrator (ESI): A Foundation ECG Model Pretrained with LLM-Enhanced Cardiological Textcited by 25
  12. 2023Medical Vision Language Pretraining: A surveycited by 24
38 more
  1. 2024Fact-Aware Multimodal Retrieval Augmentation for Accurate Medical Radiology Report Generationcited by 23
  2. 2024Read Like a Radiologist: Efficient Vision-Language Model for 3D Medical Imaging Interpretationcited by 18
  3. 2025Causally Reliable Concept Bottleneck Modelscited by 17
  4. 2024Improving Medical Multi-modal Contrastive Learning with Expert Annotationscited by 16
  5. 2024M4CXR: Exploring Multi-task Potentials of Multi-modal Large Language Models for Chest X-ray Interpretationcited by 16
  6. 2025Towards Scalable Language-Image Pre-training for 3D Medical Imagingcited by 13
  7. 2025Enhancing SAM with Efficient Prompting and Preference Optimization for Semi-supervised Medical Image Segmentationcited by 12
  8. 2025MedUnifier: Unifying Vision-and-Language Pre-training on Medical Data with Vision Generation Task using Discrete Visual Representationscited by 11
  9. 2024Advancing Text-Driven Chest X-Ray Generation with Policy-Based Reinforcement Learningcited by 10
  10. 2024PairAug: What Can Augmented Image-Text Pairs Do for Radiology?cited by 10
  11. 2024UrFound: Towards Universal Retinal Foundation Models via Knowledge-Guided Masked Modelingcited by 10
  12. 2025Bringing CLIP to the Clinic: Dynamic Soft Labels and Negation-Aware Learning for Medical Analysiscited by 10
  13. 2025Taming Vision-Language Models for Medical Image Analysis: A Comprehensive Reviewcited by 9
  14. 2024RadCLIP: Enhancing Radiologic Image Analysis through Contrastive Language-Image Pre-trainingcited by 8
  15. 2024MEDBind: Unifying Language and Multimodal Medical Data Embeddingscited by 8
  16. 2024Pre-training on High Definition X-ray Images: An Experimental Studycited by 7
  17. 2024Disease-informed Adaptation of Vision-Language Modelscited by 7
  18. 2024LADDER: Language-Driven Slice Discovery and Error Rectification in Vision Classifierscited by 7
  19. 2025Foundation Models in Medical Imaging: A Review and Outlookcited by 7
  20. 2023Masked Contrastive Reconstruction for Cross-modal Medical Image-Report Retrievalcited by 6
  21. 2025From Alignment to Synthesis: Contrastive Volumetric Grounding for Text-to-CT Generationcited by 6
  22. 2024CAR-MFL: Cross-Modal Augmentation by Retrieval for Multimodal Federated Learning with Missing Modalitiescited by 5
  23. 2024CLIP meets DINO for Tuning Zero-Shot Classifier using Unlabeled Image Collectionscited by 5
  24. 2025SilVar-Med: A Speech-Driven Visual Language Model for Explainable Abnormality Detection in Medical Imagingcited by 5
  25. 2025Mammo-FM: Breast-specific foundational model for Integrated Mammographic Diagnosis, Prognosis, and Reportingcited by 5
  26. 2024Leveraging Task-Specific Knowledge from LLM for Semi-Supervised 3D Medical Image Segmentationcited by 4
  27. 2024Position-Guided Prompt Learning for Anomaly Detection in Chest X-Rayscited by 3
  28. 2026Learning to Read Where to Look: Disease-Aware Vision-Language Pretraining for 3D CTcited by 3
  29. 2024ZALM3: Zero-Shot Enhancement of Vision-Language Alignment via In-Context Information in Multi-Turn Multimodal Medical Dialoguecited by 2
  30. 2025X2CT-CLIP: Enable Multi-Abnormality Detection in Computed Tomography from Chest Radiography via Tri-Modal Contrastive Learningcited by 2
  31. 2025A Reality Check of Vision-Language Pre-training in Radiology: Have We Progressed Using Text?cited by 2
  32. 2025Causal Disentanglement for Robust Long-tail Medical Image Generationcited by 2
  33. 2025Generate to Ground: Multimodal Text Conditioning Boosts Phrase Grounding in Medical Vision-Language Modelscited by 2
  34. 2025ProbMed: A Probabilistic Framework for Medical Multimodal Bindingcited by 2
  35. 2025A Disease-Aware Dual-Stage Framework for Chest X-ray Report Generationcited by 2
  36. 2026TextME: Bridging Unseen Modalities Through Text Descriptionscited by 2
  37. 2026A data- and compute-efficient chest X-ray foundation model beyond aggressive scalingcited by 2
  38. 2024MeDSLIP: Medical Dual-Stream Language-Image Pre-training with Pathology-Anatomy Semantic Alignmentcited by 1

50 most cited of 81 All 81 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026SentZero: An Enhanced Sentence-Centric Vision-Language Pretraining for Multi-Task Zero-Shot Chest X-Ray Analysis
  2. 2025CoCa-CXR: Contrastive Captioners Learn Strong Temporal Structures for Chest X-Ray Vision-Language Understanding
  3. 2026CXR-Retrieve: Compositional Text-to-Image Retrieval in Chest Radiography
  4. 2024Visual Prompt Engineering for Vision Language Models in Radiology
  5. 2023MedXChat: A Unified Multimodal Large Language Model Framework towards CXRs Understanding and Generation
  6. 2023Vision-Language Modelling For Radiological Imaging and Reports In The Low Data Regime
  7. 2025Benchmarking Chest X-ray Diagnosis Models Across Multinational Datasets
  8. 2023ELIXR: Towards a general purpose X-ray artificial intelligence system through alignment of large language models and radiology vision encoders