mlsanityData to 7 Oct 2026

ROCOv2: Radiology Objects in COntext Version 2, an Updated Multimodal Image Dataset

arXiv 2405.10004 ↗First posted 16 May 2024eess.IV cs.CV cs.LGcited by 54

Not on the map. Why?

Cited by 28 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Automated medical image analysis systems often require large amounts of training data with high quality labels, which are difficult and time consuming to generate. This paper introduces Radiology Object in COntext version 2 (ROCOv2), a multimodal dataset consisting of radiological images and associated medical concepts and captions extracted from the PMC Open Access subset. It is an updated version of the ROCO dataset published in 2018, and adds 35,705 new images added to PMC since 2018. It further provides manually curated concepts for imaging modalities with additional anatomical and directional concepts for X-rays. The dataset consists of 79,789 images and has been used, with minor modifications, in the concept detection and caption prediction tasks of ImageCLEFmedical Caption 2023. The dataset is suitable for training image annotation models based on image-caption pairs, or for multi-label image classification using Unified Medical Language System (UMLS) concepts provided with each image. In addition, it can serve for pre-training of medical domain models, and evaluation of deep learning models for multi-task learning.

Key facts. Cited by 54 papers in the map (data through 7 Oct 2026). Builds on 22 of its 64 references that are in the map. First posted on arXiv 16 May 2024.

Builds on

22 of 64 references.

References · 22

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  4. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  5. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  6. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,388
  7. 2020BLEURT: Learning Robust Metrics for Text Generationcited by 1,196
  8. 2017Mixed Precision Trainingcited by 1,135
  9. 2021EfficientNetV2: Smaller Models and Faster Trainingcited by 1,127
  10. 2014Show and Tell: A Neural Image Caption Generatorcited by 877
  11. 2016Bag of Tricks for Efficient Text Classificationcited by 869
  12. 2021ImageNet-21K Pretraining for the Massescited by 607
10 more
  1. 2023OpenFlamingo: An Open-Source Framework for Training Large Autoregressive Vision-Language Modelscited by 560
  2. 2023BiomedCLIP: a multimodal biomedical foundation model pretrained from fifteen million scientific image-text pairscited by 506
  3. 2021SLAKE: A Semantically-Labeled Knowledge-Enhanced Dataset for Medical Visual Question Answeringcited by 444
  4. 2023Med-Flamingo: a Multimodal Medical Few-shot Learnercited by 387
  5. 2023PMC-VQA: Visual Instruction Tuning for Medical Visual Question Answeringcited by 333
  6. 2019PadChest: A large chest x-ray image dataset with multi-label annotated reportscited by 308
  7. 2023PMC-CLIP: Contrastive Language-Image Pre-training using Biomedical Documentscited by 212
  8. 2020MedMNIST Classification Decathlon: A Lightweight AutoML Benchmark for Medical Image Analysiscited by 206
  9. 2020MedICaT: A Dataset of Medical Images, Captions, and Textual Referencescited by 71
  10. 2020Multi-domain Clinical Natural Language Processing with MedCAT: the Medical Concept Annotation Toolkitcited by 66

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 4; 2025: 22; 2026: 28.

54 papers.

Sort:

By area:

  1. 2025Lingshu: A Generalist Foundation Model for Unified Multimodal Medical Understanding and Reasoningcited by 217
  2. 2024VividMed: Vision Language Model with Versatile Visual Grounding for Medicinecited by 28
  3. 2024GMAI-VL & GMAI-VL-5.5M: A Large Vision-Language Model and A Comprehensive Multimodal Dataset Towards General Medical AIcited by 23
  4. 2025Vision Language Models in Medicinecited by 12
  5. 2026MedMO: Grounding and Understanding Multimodal Large Language Model for Medical Imagescited by 12
  6. 2025On the Robustness of Medical Vision-Language Models: Are they Truly Generalizable?cited by 10
  7. 2025Citrus-V: Advancing Medical Foundation Models with Unified Medical Image Grounding for Clinical Reasoningcited by 8
  8. 2025Toward a Vision-Language Foundation Model for Medical Data: Multimodal Dataset and Benchmarks for Vietnamese PET/CT Report Generationcited by 7
  9. 2026MedXIAOHE: A Comprehensive Recipe for Building Medical MLLMscited by 6
  10. 2025Prompt to Polyp: Medical Text-Conditioned Image Synthesis with Diffusion Modelscited by 5
  11. 2025Adapting Vision-Language Foundation Model for Next Generation Medical Ultrasound Image Analysiscited by 5
  12. 2024Light-weight Fine-tuning Method for Defending Adversarial Noise in Pre-trained Medical Vision-Language Modelscited by 4
38 more
  1. 2025Dual-Priv Pruning : Efficient Differential Private Fine-Tuning in Multimodal Large Language Modelscited by 4
  2. 2025Benchmarking Foundation Models and Parameter-Efficient Fine-Tuning for Prognosis Prediction in Medical Imagingcited by 3
  3. 2026Unveiling and Bridging the Functional Perception Gap in MLLMs: Atomic Visual Alignment and Hierarchical Evaluation via PET-Benchcited by 3
  4. 2026Do Multimodal RAG Systems Leak Data? A Comprehensive Evaluation of Membership Inference and Image Caption Retrieval Attackscited by 3
  5. 2025MedicalNarratives: Connecting Medical Vision and Language with Localized Narrativescited by 2
  6. 2025M3Ret: Unleashing Zero-shot Multimodal Medical Image Retrieval via Self-Supervisioncited by 2
  7. 2025Synthesizing High-Quality Visual Question Answering from Medical Documents with Generator-Verifier LMMscited by 2
  8. 2025Foundation Models in Biomedical Imaging: Turning Hype into Realitycited by 2
  9. 2026CT-Bench: A Benchmark for Multimodal Lesion Understanding in Computed Tomographycited by 2
  10. 2026Dialectic-Med: Mitigating Diagnostic Hallucinations via Counterfactual Adversarial Multi-Agent Debatecited by 2
  11. 2024MiniGPT-Pancreas: Multimodal Large Language Model for Pancreas Cancer Classification and Detectioncited by 1
  12. 2025MedBLIP: Fine-tuning BLIP for Medical Image Captioningcited by 1
  13. 2025HeteroRAG: A Heterogeneous Retrieval-Augmented Generation Framework for Medical Vision Language Taskscited by 1
  14. 2025QwenCLIP: Boosting Medical Vision-Language Pretraining via LLM Embeddings and Prompt tuningcited by 1
  15. 2025VL-OrdinalFormer: Vision Language Guided Ordinal Transformers for Interpretable Knee Osteoarthritis Gradingcited by 1
  16. 2026Leveraging Data to Say No: Memory Augmented Plug-and-Play Selective Predictioncited by 1
  17. 2026Beyond Classification: Dynamic Adapter Routing for Continual Multimodal Retrievalcited by 1
  18. 2026Baichuan-M4: A Clinical-Grade Medical Agent System for Continuous Carecited by 1
  19. 2026Can Experts Adapt Without Training? On Test-Time Modality Generalization in MVLMscited by 1
  20. 2025Brain Imaging Foundation Models, Are We There Yet? A Systematic Review of Foundation Models for Brain Imaging and Biomedical Researchcited by 0
  21. 2025A Semantically-Aware Relevance Measure for Content-Based Medical Image Retrieval Evaluationcited by 0
  22. 2025GRR-CoCa: Leveraging LLM Mechanisms in Multimodal Model Architecturescited by 0
  23. 2025InfiMed-Foundation: Pioneering Advanced Multimodal Medical Models with Compute-Efficient Pre-Training and Multi-Stage Fine-Tuningcited by 0
  24. 2025Cross-modal ultra-scale learning with tri-modalities of renal biopsy images for glomerular multi-disease auxiliary diagnosiscited by 0
  25. 2026Does medical specialization of VLMs enhance discriminative power?: A comprehensive investigation through feature distribution analysiscited by 0
  26. 2026How Do Inpainting Artifacts Propagate to Language?cited by 0
  27. 2026MedGPT-oss: Training a General-Purpose Vision-Language Model for Biomedicinecited by 0
  28. 2026DREAM: Dynamic Retinal Enhancement with Adaptive Multi-modal Fusion for Expert Precision Medical Report Generationcited by 0
  29. 2026Scientific Domain Knowledge Improves Vision-Language Fundus Modelscited by 0
  30. 2026MSD-Score: Multi-Scale Distributional Scoring for Reference-Free Image Caption Evaluationcited by 0
  31. 2026MEDLAYXPLAIN: Benchmarking the Expert-Lay Gap in Medical Vision-Language Modelscited by 0
  32. 2026Revealing Training Data Exposure in Vision Language Large Models via Parameter Gradientscited by 0
  33. 2026MatMMExtract: An Open-Source Pipeline for Panel-Level Extraction of Grounded Image-Text Pairs from Materials Science Literaturecited by 0
  34. 2026Explaining BiomedCLIP with Weighted Banzhaf Interactions Supported by Tree-Gram Parsingcited by 0
  35. 2026DS@GT ARC at ImageCLEFmedical 2026: Architectural Diversity for Concept Detection and Foundation-Model Scaling for Caption Prediction in Medical Image Analysiscited by 0
  36. 2026Clinically Structured Surrogate Rewards for Post-SFT Medical Image Captioningcited by 0
  37. 2026MedUAG: Unified Understanding and Generation for Medical Multimodal Modelscited by 0
  38. 2026Towards Clinically Faithful Medical Image Captioning via Enhanced Vision-Language Alignmentcited by 0

50 most cited of 54 All 54 (JSON)