# A Vision-language Framework for Comparative Reasoning in Radiology arXiv: https://arxiv.org/abs/2606.06407 ยท HTML: https://mlsanity.com/paper/2606.06407/ ## Key facts - Cited by 1 papers in the map (data through 14 Oct 2026). - Builds on 36 of its 62 references that are in the map. - First posted on arXiv 4 Jun 2026. ## Builds on (most cited first, 30 of 36) - [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024) - [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018) - [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019) - [CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparison](https://mlsanity.com/paper/1901.07031/) (2019) - [Scaling Vision with Sparse Mixture of Experts](https://mlsanity.com/paper/2106.05974/) (2021) - [MedCLIP: Contrastive Learning from Unpaired Medical Images and Text](https://mlsanity.com/paper/2210.10163/) (2022) - [CheXNet: Radiologist-Level Pneumonia Detection on Chest X-Rays with Deep Learning](https://mlsanity.com/paper/1711.05225/) (2017) - [MedGemma Technical Report](https://mlsanity.com/paper/2507.05201/) (2025) - [Deep metric learning using Triplet network](https://mlsanity.com/paper/1412.6622/) (2014) - [Generating Radiology Reports via Memory-driven Transformer](https://mlsanity.com/paper/2010.16056/) (2020) - [AMOS: A Large-Scale Abdominal Multi-Organ Benchmark for Versatile Medical Image Segmentation](https://mlsanity.com/paper/2206.08023/) (2022) - [Lingshu: A Generalist Foundation Model for Unified Multimodal Medical Understanding and Reasoning](https://mlsanity.com/paper/2506.07044/) (2025) - [On the Automatic Generation of Medical Imaging Reports](https://mlsanity.com/paper/1711.08195/) (2017) - [PMC-CLIP: Contrastive Language-Image Pre-training using Biomedical Documents](https://mlsanity.com/paper/2303.07240/) (2023) - [Generalist Foundation Models from a Multimodal Dataset for 3D Computed Tomography](https://mlsanity.com/paper/2403.17834/) (2024) - [MAIRA-2: Grounded Radiology Report Generation](https://mlsanity.com/paper/2406.04449/) (2024) - [CheXpert Plus: Augmenting a Large Chest X-ray Dataset with Text Radiology Reports, Patient Demographics and Additional Image Formats](https://mlsanity.com/paper/2405.19538/) (2024) - [Knowledge-driven Encode, Retrieve, Paraphrase for Medical Image Report Generation](https://mlsanity.com/paper/1903.10122/) (2019) - [Hulu-Med: A Transparent Generalist Model towards Holistic Medical Vision-Language Understanding](https://mlsanity.com/paper/2510.08668/) (2025) - [BioMedGPT: Open Multimodal Generative Pre-trained Transformer for BioMedicine](https://mlsanity.com/paper/2308.09442/) (2023) - [RaTEScore: A Metric for Radiology Report Generation](https://mlsanity.com/paper/2406.16845/) (2024) - [Large-Vocabulary Segmentation for Medical Images with Text Prompts](https://mlsanity.com/paper/2312.17183/) (2023) - [Expert Knowledge-Aware Image Difference Graph Representation Learning for Difference-Aware Medical Visual Question Answering](https://mlsanity.com/paper/2307.11986/) (2023) - [Utilizing Longitudinal Chest X-Rays and Reports to Pre-Fill Radiology Reports](https://mlsanity.com/paper/2306.08749/) (2023) - [HERGen: Elevating Radiology Report Generation with Longitudinal Data](https://mlsanity.com/paper/2407.15158/) (2024) - [BIMCV-R: A Landmark Dataset for 3D CT Text-Image Retrieval](https://mlsanity.com/paper/2403.15992/) (2024) - [3D-RAD: A Comprehensive 3D Radiology Med-VQA Dataset with Multi-Temporal Analysis and Diverse Diagnostic Tasks](https://mlsanity.com/paper/2506.11147/) (2025) - [Controllable Chest X-Ray Report Generation from Longitudinal Representations](https://mlsanity.com/paper/2310.05881/) (2023) - [BioLORD: Learning Ontological Representations from Definitions (for Biomedical Concepts and their Textual Descriptions)](https://mlsanity.com/paper/2210.11892/) (2022) - [Ultrasound Report Generation with Cross-Modality Feature Alignment via Unsupervised Guidance](https://mlsanity.com/paper/2406.00644/) (2024) ## Built on by (most cited first, 1 of 1) - [Volumetric Radiology AI in the Era of Multimodal Large Language Models](https://mlsanity.com/paper/2608.20549/) (2026) ## Abstract (arXiv) Medical imaging artificial intelligence has achieved strong performance in isolated image interpretation, but remains poorly aligned with radiological practice, where diagnosis and follow-up rely on comparison across prior studies and analogous reference cases. Here we formulate radiological comparison as an entity-aware cross-image reasoning problem and introduce a framework that supports both reference-case retrieval and temporal comparative interpretation. We construct MedReCo-DB, a large-scale comparative imaging resource derived from routine image-report pairs, comprising more than 690,000 images from over 160,000 patients across eight institutions, four countries and seven imaging modalities. Reports are decomposed into anatomical structures, abnormal findings and pathological conditions to provide supervision for entity-conditioned retrieval and comparative visual question answering. Using this resource, we develop MedReCo, an entity-aware visual encoder for controllable retrieval of clinically analogous cases, and MedReCo-VLM, a vision--language extension for generative interpretation of interval change. Across internal, external and cross-center evaluations, MedReCo achieved the highest Recall@1 in all 12 internal retrieval settings and improved external retrieval by a mean of 6.0 percentage points. In clinically confusable differential groups, it consistently outperformed the strongest baselines. MedReCo-VLM achieved the best performance across all comparative generation evaluations and improved longitudinal follow-up accuracy by 14.5-46.5 percentage points on chest radiographs and 13.0-27.9 percentage points on CT. These findings suggest that entity-aware comparative reasoning can be learned from routine clinical data at scale and may provide a more clinically aligned foundation for medical imaging AI.