# BIMCV-R: A Landmark Dataset for 3D CT Text-Image Retrieval

arXiv: https://arxiv.org/abs/2403.15992 · HTML: https://mlsanity.com/paper/2403.15992/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 40 references that are in the map.
- First posted on arXiv 24 Mar 2024.

## Builds on (most cited first, 20 of 20)

- [Swin UNETR: Swin Transformers for Semantic Segmentation of Brain Tumors in MRI Images](https://mlsanity.com/paper/2201.01266/) (2022)
- [CLIP4Clip: An Empirical Study of CLIP for End to End Video Clip Retrieval](https://mlsanity.com/paper/2104.08860/) (2021)
- [MedCLIP: Contrastive Learning from Unpaired Medical Images and Text](https://mlsanity.com/paper/2210.10163/) (2022)
- [Quilt-1M: One Million Image-Text Pairs for Histopathology](https://mlsanity.com/paper/2306.11207/) (2023)
- [LViT: Language meets Vision Transformer in Medical Image Segmentation](https://mlsanity.com/paper/2206.14718/) (2022)
- [BIMCV COVID-19+: a large annotated dataset of RX and CT images from COVID-19 patients](https://mlsanity.com/paper/2006.01174/) (2020)
- [Large Language Models for Forecasting and Anomaly Detection: A Systematic Literature Review](https://mlsanity.com/paper/2402.10350/) (2024)
- [Generative Text-Guided 3D Vision-Language Pretraining for Unified Medical Image Segmentation](https://mlsanity.com/paper/2306.04811/) (2023)
- [M-FLAG: Medical Vision-Language Pre-training with Frozen Language Models and Latent Space Geometry Optimization](https://mlsanity.com/paper/2307.08347/) (2023)
- [StegaNeRF: Embedding Invisible Information within Neural Radiance Fields](https://mlsanity.com/paper/2212.01602/) (2022)
- [Particle Filter SLAM for Vehicle Localization](https://mlsanity.com/paper/2402.07429/) (2024)
- [Adaptive Ensembles of Fine-Tuned Transformers for LLM-Generated Text Detection](https://mlsanity.com/paper/2403.13335/) (2024)
- [Self-Supervised Neuron Segmentation with Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2310.04148/) (2023)
- [Learning Multiscale Consistency for Self-supervised Electron Microscopy Instance Segmentation](https://mlsanity.com/paper/2308.09917/) (2023)
- [Hierarchical Deep Network with Uncertainty-aware Semi-supervised Learning for Vessel Segmentation](https://mlsanity.com/paper/2105.14732/) (2021)
- [News Recommendation with Attention Mechanism](https://mlsanity.com/paper/2402.07422/) (2024)
- [T3D: Advancing 3D Medical Vision-Language Pre-training by Learning Multi-View Visual Consistency](https://mlsanity.com/paper/2312.01529/) (2023)
- [Generator Versus Segmentor: Pseudo-healthy Synthesis](https://mlsanity.com/paper/2009.05722/) (2020)
- [3D-MIR: A Benchmark and Empirical Study on 3D Medical Image Retrieval in Radiology](https://mlsanity.com/paper/2311.13752/) (2023)
- [Data-Driven Filter Design in FBP: Transforming CT Reconstruction with Trainable Fourier Series](https://mlsanity.com/paper/2401.16039/) (2024)

## Built on by (most cited first, 30 of 30)

- [OFFSET: Segmentation-based Focus Shift Revision for Composed Image Retrieval](https://mlsanity.com/paper/2507.05631/) (2025)
- [HUD: Hierarchical Uncertainty-Aware Disambiguation Network for Composed Video Retrieval](https://mlsanity.com/paper/2512.02792/) (2025)
- [CU-Net: a U-Net architecture for efficient brain-tumor segmentation on BraTS 2019 dataset](https://mlsanity.com/paper/2406.13113/) (2024)
- [MSP-MVS: Multi-Granularity Segmentation Prior Guided Multi-View Stereo](https://mlsanity.com/paper/2407.19323/) (2024)
- [Mask Factory: Towards High-quality Synthetic Data Generation for Dichotomous Image Segmentation](https://mlsanity.com/paper/2412.19080/) (2024)
- [DVP-MVS: Synergize Depth-Edge and Visibility Prior for Multi-View Stereo](https://mlsanity.com/paper/2412.11578/) (2024)
- [CRISP-SAM2: SAM2 with Cross-Modal Interaction and Semantic Prompting for Multi-Organ Segmentation](https://mlsanity.com/paper/2506.23121/) (2025)
- [Research on Image Recognition Technology Based on Multimodal Deep Learning](https://mlsanity.com/paper/2405.03091/) (2024)
- [DenoMamba: A fused state-space model for low-dose CT denoising](https://mlsanity.com/paper/2409.13094/) (2024)
- [A Survey of Medical Vision-and-Language Applications and Their Techniques](https://mlsanity.com/paper/2411.12195/) (2024)
- [SED-MVS: Segmentation-Driven and Edge-Aligned Deformation Multi-View Stereo with Depth Restoration and Occlusion Constraint](https://mlsanity.com/paper/2503.13721/) (2025)
- [Argus: Benchmarking and Enhancing Vision-Language Models for 3D Radiology Report Generation](https://mlsanity.com/paper/2406.07146/) (2024)
- [Taming Vision-Language Models for Medical Image Analysis: A Comprehensive Review](https://mlsanity.com/paper/2506.18378/) (2025)
- [TokenUnify: Scaling Up Autoregressive Pretraining for Neuron Segmentation](https://mlsanity.com/paper/2405.16847/) (2024)
- [CoC-VLA: Delving into Adversarial Domain Transfer for Explainable Autonomous Driving via Chain-of-Causality Visual-Language-Action Model](https://mlsanity.com/paper/2511.19914/) (2025)
- [From large language models to multimodal AI: A scoping review on the potential of generative AI in medicine](https://mlsanity.com/paper/2502.09242/) (2025)
- [M3Retrieve: Benchmarking Multimodal Retrieval for Medicine](https://mlsanity.com/paper/2510.06888/) (2025)
- [UniCompress: Enhancing Multi-Data Medical Image Compression with Knowledge Distillation](https://mlsanity.com/paper/2405.16850/) (2024)
- [Rethinking Whole-Body CT Image Interpretation: An Abnormality-Centric Approach](https://mlsanity.com/paper/2506.03238/) (2025)
- [M3Ret: Unleashing Zero-shot Multimodal Medical Image Retrieval via Self-Supervision](https://mlsanity.com/paper/2509.01360/) (2025)
- [SigVLP: Sigmoid Volume-Language Pre-Training for Self-Supervised CT-Volume Adaptive Representation Learning](https://mlsanity.com/paper/2602.21735/) (2026)
- [HSENet: Hybrid Spatial Encoding Network for 3D Medical Vision-Language Understanding](https://mlsanity.com/paper/2506.09634/) (2025)
- [RadiomicsRetrieval: A Customizable Framework for Medical Image Retrieval Using Radiomics Features](https://mlsanity.com/paper/2507.08546/) (2025)
- [Dual form Complementary Masking for Domain-Adaptive Image Segmentation](https://mlsanity.com/paper/2507.12008/) (2025)
- [ACM Multimedia Grand Challenge on ENT Endoscopy Analysis](https://mlsanity.com/paper/2508.04801/) (2025)
- [INFORM-CT: INtegrating LLMs and VLMs FOR Incidental Findings Management in Abdominal CT](https://mlsanity.com/paper/2512.14732/) (2025)
- [A Vision-language Framework for Comparative Reasoning in Radiology](https://mlsanity.com/paper/2606.06407/) (2026)
- [brat: Aligned Multi-View Embeddings for Brain MRI Analysis](https://mlsanity.com/paper/2512.18679/) (2025)
- [nnFoundation: 3D Foundation Models for Radiology](https://mlsanity.com/paper/2609.26924/) (2026)
- [CAPTOR: a vision-language foundation model for Chest-Abdomen-Pelvis Oncologic CT](https://mlsanity.com/paper/2610.14226/) (2026)

## Abstract (arXiv)

The burgeoning integration of 3D medical imaging into healthcare has led to a substantial increase in the workload of medical professionals. To assist clinicians in their diagnostic processes and alleviate their workload, the development of a robust system for retrieving similar case studies presents a viable solution. While the concept holds great promise, the field of 3D medical text-image retrieval is currently limited by the absence of robust evaluation benchmarks and curated datasets. To remedy this, our study presents a groundbreaking dataset, BIMCV-R, which includes an extensive collection of 8,069 3D CT volumes, encompassing over 2 million slices, paired with their respective radiological reports. Expanding upon the foundational work of our dataset, we craft a retrieval strategy, MedFinder. This approach employs a dual-stream network architecture, harnessing the potential of large language models to advance the field of medical image retrieval beyond existing text-image retrieval solutions. It marks our preliminary step towards developing a system capable of facilitating text-to-image, image-to-text, and keyword-based retrieval tasks. Our project is available at urlhttps://huggingface.co/datasets/cyd0806/BIMCV-R.