mlsanityData to 7 Oct 2026

CXR-Retrieve: Compositional Text-to-Image Retrieval in Chest Radiography

arXiv 2607.27779 ↗First posted 30 Jul 2026cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Large chest radiography archives are difficult to search because most studies are paired only with free-text reports rather than structured clinical annotations. Vision-language models offer a natural interface for text-to-image retrieval, but current biomedical models are primarily optimized for report-to-image matching rather than for satisfying short clinical search queries. This creates an objective mismatch: a model may retrieve images related to words in the query while failing to satisfy the full clinical constraint, especially for conjunctions and negations such as ``atelectasis and no pneumonia.'' We introduce CXR-Retrieve, a structured benchmark for compositional chest X-ray text-to-image retrieval. The benchmark contains 5,159 test images from the official test-split of MIMIC-CXR-JPG and 145 textual queries spanning single and conjunction findings, both positive and negative. Relevance is defined by whether a retrieved image satisfies all asserted pathology constraints, rather than by whether it matches a paired report. We further propose a label-aware contrastive fine-tuning objective for clinical retrieval. Our method attracts image-text pairs with compatible asserted pathology constraints, including shared confirmed absences, while explicitly repelling contradictory pairs. Starting from the in-domain CXR-CLIP checkpoint, our method improves Precision@5 over CXR-CLIP by 8.5 percentage points on two-pathology conjunctions and by 22.0 percentage points on negation queries. These results show that reliable chest X-ray retrieval requires training objectives that model not only which findings are mentioned, but also how they are clinically asserted.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 20 of its 33 references that are in the map. First posted on arXiv 30 Jul 2026.

Builds on

20 of 33 references.

References · 20

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  3. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  4. 2019CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparisoncited by 1,410
  5. 2017A Survey on Deep Learning in Medical Image Analysiscited by 1,182
  6. 2022Reproducible scaling laws for contrastive language-image learningcited by 1,163
  7. 2017ChestX-ray8: Hospital-scale Chest X-ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseasescited by 969
  8. 2020Contrastive Learning of Medical Visual Representations from Paired Images and Textcited by 636
  9. 2022When and why vision-language models behave like bags-of-words, and what to do about it?cited by 603
  10. 2019MIMIC-CXR-JPG, a large publicly available database of labeled chest radiographscited by 590
  11. 2022Winoground: Probing Vision and Language Models for Visio-Linguistic Compositionalitycited by 523
  12. 2023BiomedCLIP: a multimodal biomedical foundation model pretrained from fifteen million scientific image-text pairscited by 506
8 more
  1. 2017NegBio: a high-performance tool for negation and uncertainty detection in radiology reportscited by 79
  2. 2025Vision-Language Models Do Not Understand Negationcited by 69
  3. 2023COLA: A Benchmark for Compositional Text-to-image Retrievalcited by 53
  4. 2023Multimodal Image-Text Matching Improves Retrieval-based Chest X-Ray Report Generationcited by 37
  5. 2017Medical Image Retrieval using Deep Convolutional Neural Networkcited by 33
  6. 2023Semi-Supervised Semantic Segmentation via Marginal Contextual Informationcited by 8
  7. 2025CARES: Context-Aware Resolution Selector for VLMscited by 5
  8. 2026Look Where It Matters: High-Resolution Crops Retrieval for Efficient VLMscited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Exploring the Capabilities of Large Language Model Encoders for Image-Text Retrieval in Chest X-rays
  2. 2025CoCa-CXR: Contrastive Captioners Learn Strong Temporal Structures for Chest X-Ray Vision-Language Understanding
  3. 2026Multi-task Cross-modal Learning for Chest X-ray Image Retrieval
  4. 2026MedProbCLIP: Probabilistic Adaptation of Vision-Language Foundation Model for Reliable Radiograph-Report Retrieval
  5. 2023CXR-CLIP: Toward Large Scale Chest X-ray Language-Image Pre-training
  6. 2026MC-CXR: A Multi-Context Chest X-ray Benchmark for Context-Induced Disruption in Vision-Language Models
  7. 2026SentZero: An Enhanced Sentence-Centric Vision-Language Pretraining for Multi-Task Zero-Shot Chest X-Ray Analysis
  8. 2025A Generative Framework for Bidirectional Image-Report Understanding in Chest Radiography