mlsanityData to 7 Oct 2026

Large-scale Long-tailed Disease Diagnosis on Radiology Images

arXiv 2312.16151 ↗First posted 26 Dec 2023cs.CVcited by 17

Not on the map. Why?

Abstract (arXiv)

Developing a generalist radiology diagnosis system can greatly enhance clinical diagnostics. In this paper, we introduce RadDiag, a foundational model supporting 2D and 3D inputs across various modalities and anatomies, using a transformer-based fusion module for comprehensive disease diagnosis. Due to patient privacy concerns and the lack of large-scale radiology diagnosis datasets, we utilize high-quality, clinician-reviewed radiological images available online with diagnosis labels. Our dataset, RP3D-DiagDS, contains 40,936 cases with 195,010 scans covering 5,568 disorders (930 unique ICD-10-CM codes). Experimentally, our RadDiag achieves 95.14% AUC on internal evaluation with the knowledge-enhancement strategy. Additionally, RadDiag can be zero-shot applied or fine-tuned to external diagnosis datasets sourced from various hospitals, demonstrating state-of-the-art results. In conclusion, we show that publicly shared medical data on the Internet is a tremendous and valuable resource that can potentially support building a generalist AI for healthcare.

Key facts. Cited by 17 papers in the map (data through 7 Oct 2026). Builds on 54 of its 107 references that are in the map. First posted on arXiv 26 Dec 2023.

Builds on

54 of 107 references.

References · 54

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2023GPT-4 Technical Reportcited by 18,132
  4. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  5. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  6. 2016Identity Mappings in Deep Residual Networkscited by 1,960
  7. 2019CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparisoncited by 1,410
  8. 2019LVIS: A Dataset for Large Vocabulary Instance Segmentationcited by 1,215
  9. 2017ChestX-ray8: Hospital-scale Chest X-ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseasescited by 969
  10. 2020Contrastive Learning of Medical Visual Representations from Paired Images and Textcited by 636
  11. 2019Score-CAM: Score-Weighted Visual Explanations for Convolutional Neural Networkscited by 575
  12. 2021TransMIL: Transformer based Correlated Multiple Instance Learning for Whole Slide Image Classificationcited by 549
38 more
  1. 2023BiomedCLIP: a multimodal biomedical foundation model pretrained from fifteen million scientific image-text pairscited by 506
  2. 2022MONAI: An open-source framework for deep learning in healthcarecited by 438
  3. 2023Med-Flamingo: a Multimodal Medical Few-shot Learnercited by 387
  4. 2017Learning Spatio-Temporal Representation with Pseudo-3D Residual Networkscited by 386
  5. 2020Asymmetric Loss For Multi-Label Classificationcited by 379
  6. 2023PMC-VQA: Visual Instruction Tuning for Medical Visual Question Answeringcited by 333
  7. 2019PadChest: A large chest x-ray image dataset with multi-label annotated reportscited by 308
  8. 2023Towards Generalist Biomedical AIcited by 286
  9. 2020VinDr-CXR: An open dataset of chest X-rays with radiologist's annotationscited by 236
  10. 2023PMC-CLIP: Contrastive Language-Image Pre-training using Biomedical Documentscited by 212
  11. 2020Can AI help in screening Viral and COVID-19 pneumonia?cited by 200
  12. 2020Eigen-CAM: Class Activation Map using Principal Componentscited by 188
  13. 2023MedCPT: Contrastive Pre-trained Transformers with Large-scale PubMed Search Logs for Zero-shot Biomedical Information Retrievalcited by 182
  14. 2023Towards Generalist Foundation Model for Radiology by Leveraging Web-scale 2D&3D Medical Datacited by 176
  15. 2023MedKLIP: Medical Knowledge Enhanced Language-Image Pre-Training in Radiologycited by 149
  16. 2023Knowledge-enhanced Visual-Language Pre-training on Chest Radiology Imagescited by 128
  17. 2020MosMedData: Chest CT Scans With COVID-19 Related Findings Datasetcited by 103
  18. 2022mmFormer: Multimodal Medical Transformer for Incomplete Multimodal Learning of Brain Tumor Segmentationcited by 94
  19. 2021TransMed: Transformers Advance Multi-modal Medical Image Classificationcited by 85
  20. 2022A graph-transformer for whole slide image classificationcited by 84
  21. 2022VinDr-Mammo: A large-scale benchmark dataset for computer-aided diagnosis in full-field digital mammographycited by 83
  22. 2021MIA-COV19D: COVID-19 Detection through 3-D Chest CT Image Analysiscited by 72
  23. 2020Machine-Learning-Based Multiple Abnormality Prediction with Large-Scale Chest Computed Tomography Volumescited by 66
  24. 2021Is it Time to Replace CNNs with Transformers for Medical Images?cited by 60
  25. 2017Residual and Plain Convolutional Neural Networks for 3D Brain MRI Classificationcited by 47
  26. 2022Equalized Focal Loss for Dense Long-Tailed Object Detectioncited by 46
  27. 2020ChestX-Det10: Chest X-ray Dataset on Detection of Thoracic Abnormalitiescited by 37
  28. 2019A Survey on Biomedical Image Captioningcited by 30
  29. 2019Intracranial Hemorrhage Segmentation Using Deep Convolutional Modelcited by 29
  30. 2019LNDb: A Lung Nodule Database on Computed Tomographycited by 27
  31. 2021COVID-VIT: Classification of COVID-19 from CT chest images based on vision transformer modelscited by 27
  32. 2020POCOVID-Net: Automatic Detection of COVID-19 From a New Lung Ultrasound Imaging Dataset (POCUS)cited by 25
  33. 2017DeepLesion: Automated Deep Mining, Categorization and Detection of Significant Radiology Image Findings using Large-Scale Clinical Lesion Annotationscited by 23
  34. 2019Radial-Based Undersampling for Imbalanced Data Classificationcited by 22
  35. 2021COVID-Net CXR-2: An Enhanced Deep Convolutional Neural Network Design for Detection of COVID-19 Cases from Chest X-ray Imagescited by 22
  36. 2021Vision Transformer for Classification of Breast Ultrasound Imagescited by 21
  37. 2021VinDr-SpineXR: A deep learning framework for spinal lesions detection and classification from radiographscited by 19
  38. 2022PediCXR: An open, large-scale chest radiograph dataset for interpretation of common thoracic diseases in childrencited by 19

50 most cited of 54 All 54 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 3; 2025: 8; 2026: 5.

17 papers.

Sort:

By area:

  1. 2024RaTEScore: A Metric for Radiology Report Generationcited by 65
  2. 2023Large-Vocabulary Segmentation for Medical Images with Text Promptscited by 60
  3. 2025SurgVLM: A Large Vision-Language Model and Systematic Evaluation Benchmark for Surgical Intelligencecited by 30
  4. 2024Large Language Model Benchmarks in Medical Taskscited by 16
  5. 2025Chiron-o1: Igniting Multimodal Large Language Models towards Generalizable Medical Reasoning via Mentor-Intern Collaborative Searchcited by 16
  6. 2025CXR-LT 2024: A MICCAI challenge on long-tailed, multi-label, and zero-shot disease classification from chest X-raycited by 10
  7. 2024How Well Can Modern LLMs Act as Agent Cores in Radiology Environments?cited by 9
  8. 2025Rethinking Whole-Body CT Image Interpretation: An Abnormality-Centric Approachcited by 2
  9. 2025R2Seg: Training-Free OOD Medical Tumor Segmentation via Anatomical Reasoning and Statistical Rejectioncited by 1
  10. 2026ProFound: A moderate-sized vision foundation model for multi-task prostate imagingcited by 1
  11. 2026ASAP: Advancing Medical Volumetric Representation Learning with Anatomy-aware Semantically-adaptive Pre-trainingcited by 1
  12. 2025Empowering Functional Neuroimaging: A Pre-trained Generative Framework for Unified Representation of Neural Signalscited by 0
5 more
  1. 2025Ontology-Based Concept Distillation for Radiology Report Retrieval and Labelingcited by 0
  2. 2025From Generic to Specialized: A Subspecialty Diagnostic System Powered by Self-Supervised Learning for Cervical Histopathologycited by 0
  3. 2026Early Intervention for VFM-based Multimodal Medical Image Classificationcited by 0
  4. 2026Dementia Etiology Diagnosis via Collaborative Meta Knowledge Enhancementcited by 0
  5. 2026Generalized Biomedicine Discoverycited by 0