mlsanityData to 7 Oct 2026

ChatCAD: Interactive Computer-Aided Diagnosis on Medical Image using Large Language Models

arXiv 2302.07257 ↗First posted 14 Feb 2023cs.CV eess.IVcited by 116

Not on the map. Why?

Cited by 25 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Large language models (LLMs) have recently demonstrated their potential in clinical applications, providing valuable medical knowledge and advice. For example, a large dialog LLM like ChatGPT has successfully passed part of the US medical licensing exam. However, LLMs currently have difficulty processing images, making it challenging to interpret information from medical images, which are rich in information that supports clinical decisions. On the other hand, computer-aided diagnosis (CAD) networks for medical images have seen significant success in the medical field by using advanced deep-learning algorithms to support clinical decision-making. This paper presents a method for integrating LLMs into medical-image CAD networks. The proposed framework uses LLMs to enhance the output of multiple CAD networks, such as diagnosis networks, lesion segmentation networks, and report generation networks, by summarizing and reorganizing the information presented in natural language text format. The goal is to merge the strengths of LLMs' medical domain knowledge and logical reasoning with the vision understanding capability of existing medical-image CAD models to create a more user-friendly and understandable system for patients compared to conventional CAD systems. In the future, LLM's medical knowledge can be also used to improve the performance of vision-based medical-image CAD models.

Key facts. Cited by 116 papers in the map (data through 7 Oct 2026). Builds on 29 of its 36 references that are in the map. First posted on arXiv 14 Feb 2023.

Builds on

29 of 36 references.

References · 29

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2020Language Models are Few-Shot Learnerscited by 32,199
  5. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  6. 2022Training language models to follow instructions with human feedbackcited by 15,712
  7. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  8. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  9. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  10. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  11. 2019BioBERT: a pre-trained biomedical language representation model for biomedical text miningcited by 1,935
  12. 2022Large Language Models Encode Clinical Knowledgecited by 1,807
17 more
  1. 2019CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparisoncited by 1,410
  2. 2020Domain-Specific Language Model Pretraining for Biomedical Natural Language Processingcited by 996
  3. 2019Publicly Available Clinical BERT Embeddingscited by 995
  4. 2021Multimodal Few-Shot Learning with Frozen Language Modelscited by 657
  5. 2017Deep Learning for Medical Image Analysiscited by 399
  6. 2020Generating Radiology Reports via Memory-driven Transformercited by 363
  7. 2020CheXbert: Combining Automatic Labelers and Expert Annotations for Accurate Radiology Report Labeling Using BERTcited by 329
  8. 2016Image Captioning with Semantic Attentioncited by 260
  9. 2019Image Captioning: Transforming Objects into Wordscited by 171
  10. 2020When Radiology Report Generation Meets Knowledge Graphcited by 144
  11. 2019Clinically Accurate Chest X-Ray Report Generationcited by 138
  12. 2022Improving Chest X-Ray Report Generation by Leveraging Warm Startingcited by 93
  13. 2022Cross-modal Prototype Driven Network for Radiology Report Generationcited by 58
  14. 2022Follow My Eye: Using Gaze to Supervise Computer-Aided Diagnosiscited by 38
  15. 2020Weakly Supervised Lesion Localization With Probabilistic-CAM Poolingcited by 35
  16. 2023RCPS: Rectified Contrastive Pseudo Supervision for Semi-Supervised Medical Image Segmentationcited by 27
  17. 2020Dual-Sampling Attention Network for Diagnosis of COVID-19 from Community Acquired Pneumoniacited by 26

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 49; 2024: 39; 2025: 20; 2026: 8.

116 papers.

Sort:

By area:

  1. 2023Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decodingcited by 671
  2. 2023PMC-VQA: Visual Instruction Tuning for Medical Visual Question Answeringcited by 333
  3. 2023Analyzing and Mitigating Object Hallucination in Large Vision-Language Modelscited by 314
  4. 2023A Survey of Large Language Models for Healthcare: from Data, Technology, and Applications to Accountability and Ethicscited by 162
  5. 2023Summary of ChatGPT-Related Research and Perspective Towards the Future of Large Language Modelscited by 152
  6. 2023On the Challenges and Perspectives of Foundation Models for Medical Image Analysiscited by 135
  7. 2023Domain Specialization as the Key to Make Large Language Models Disruptive: A Comprehensive Surveycited by 134
  8. 2023DoctorGLM: Fine-tuning your Chinese Doctor is not a Herculean Taskcited by 130
  9. 2023AugGPT: Leveraging ChatGPT for Text Data Augmentationcited by 128
  10. 2023DeID-GPT: Zero-shot Medical Text De-Identification by GPT-4cited by 128
  11. 2023Large language models in healthcare and medical domain: A reviewcited by 126
  12. 2023A Survey of Large Language Models in Medicine: Progress, Application, and Challengecited by 123
38 more
  1. 2023Opportunities and Challenges for ChatGPT and Large Language Models in Biomedicine and Healthcited by 118
  2. 2024Self-Introspective Decoding: Alleviating Hallucinations for Large Vision-Language Modelscited by 90
  3. 2024MLLM can see? Dynamic Correction Decoding for Hallucination Mitigationcited by 87
  4. 2023Foundational Models in Medical Imaging: A Comprehensive Survey and Future Visioncited by 74
  5. 2023Thread of Thought Unraveling Chaotic Contextscited by 71
  6. 2023Large AI Models in Health Informatics: Applications, Challenges, and the Futurecited by 61
  7. 2024Fairness in Large Language Models: A Taxonomic Surveycited by 57
  8. 2023When Brain-inspired AI Meets AGIcited by 54
  9. 2024Decoding Compressed Trust: Scrutinizing the Trustworthiness of Efficient LLMs Under Compressioncited by 54
  10. 2023Prompt Engineering for Healthcare: Methodologies and Applicationscited by 52
  11. 2023ECG-QA: A Comprehensive Question Answering Dataset Combined With Electrocardiogramcited by 52
  12. 2023MedBLIP: Bootstrapping Language-Image Pre-training from 3D Medical Images and Textscited by 51
  13. 2023Evaluating Large Language Models on a Highly-specialized Topic, Radiation Oncology Physicscited by 49
  14. 2023SurgicalGPT: End-to-End Language-Vision GPT for Visual Question Answering in Surgerycited by 48
  15. 2025ClearSight: Visual Signal Enhancement for Object Hallucination Mitigation in Multimodal Large language Modelscited by 40
  16. 2024Prompt Stealing Attacks Against Large Language Modelscited by 39
  17. 2023ChatCAD+: Towards a Universal and Reliable Interactive CAD using LLMscited by 35
  18. 2024LLMs Meet Multimodal Generation and Editing: A Surveycited by 35
  19. 2024A Survey on Medical Large Language Models: Technology, Application, Trustworthiness, and Future Directionscited by 34
  20. 2024GSCo: Towards Generalizable AI in Medicine via Generalist-Specialist Collaborationcited by 33
  21. 2023Artificial General Intelligence for Medical Imaging Analysiscited by 32
  22. 2023Interactive Natural Language Processingcited by 31
  23. 2024AutoHallusion: Automatic Generation of Hallucination Benchmarks for Vision-Language Modelscited by 30
  24. 2023Advancing Medical Imaging with Language Models: A Journey from N-grams to ChatGPTcited by 28
  25. 2024On Large Visual Language Models for Medical Imaging Analysis: An Empirical Studycited by 28
  26. 2024TC-RAG:Turing-Complete RAG's Case study on Medical LLM Systemscited by 28
  27. 2024Alleviating Hallucination in Large Vision-Language Models with Active Retrieval Augmentationcited by 27
  28. 2023ChatABL: Abductive Learning via Natural Language Interaction with ChatGPTcited by 26
  29. 2023Longitudinal Data and a Semantic Similarity Reward for Chest X-Ray Report Generationcited by 24
  30. 2025Attention Reallocation: Towards Zero-cost and Controllable Hallucination Mitigation of MLLMscited by 23
  31. 2023Exploring New Frontiers in Agricultural NLP: Investigating the Potential of Large Language Models for Food Applicationscited by 21
  32. 2023MuseChat: A Conversational Music Recommendation System for Videoscited by 21
  33. 2023Bias and Fairness in Chatbots: An Overviewcited by 20
  34. 2024Guiding Clinical Reasoning with Large Language Models via Knowledge Seedscited by 19
  35. 2023SkinGPT-4: An Interactive Dermatology Diagnostic System with Visual Large Language Modelcited by 18
  36. 2024ProtChatGPT: Towards Understanding Proteins with Large Language Modelscited by 18
  37. 2023Read, Diagnose and Chat: Towards Explainable and Interactive LLMs-Augmented Depression Detection in Social Mediacited by 16
  38. 2024Large Language Models in Wireless Application Design: In-Context Learning-enhanced Automatic Network Intrusion Detectioncited by 16

50 most cited of 116 All 116 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Evaluating LLM -- Generated Multimodal Diagnosis from Medical Images and Symptom Analysis
  2. 2023LLM-CXR: Instruction-Finetuned LLM for CXR Image Understanding and Generation
  3. 2025A Medical Multimodal Diagnostic Framework Integrating Vision-Language Models and Logic Tree Reasoning
  4. 2026MedReaMM: Evaluating Large Multimodal Models on Expert-Level Clinical Diagnostic Synthesis
  5. 2025Medical Knowledge Intervention Prompt Tuning for Medical Image Classification
  6. 2026A visual large language foundational model for medical image recognition using clinician-contributed online resources
  7. 2025MedAtlas: Evaluating LLMs for Multi-Round, Multi-Task Medical Reasoning Across Diverse Imaging Modalities and Clinical Text
  8. 2025Computed Tomography Visual Question Answering with Cross-modal Feature Graphing