mlsanityData to 7 Oct 2026

Multimodal Healthcare AI: Identifying and Designing Clinically Relevant Vision-Language Applications for Radiology

arXiv 2402.14252 ↗First posted 22 Feb 2024cs.HCcited by 45

Not on the map. Why?

Abstract (arXiv)

Recent advances in AI combine large language models (LLMs) with vision encoders that bring forward unprecedented technical capabilities to leverage for a wide range of healthcare applications. Focusing on the domain of radiology, vision-language models (VLMs) achieve good performance results for tasks such as generating radiology findings based on a patient's medical image, or answering visual questions (e.g., 'Where are the nodules in this chest X-ray?'). However, the clinical utility of potential applications of these capabilities is currently underexplored. We engaged in an iterative, multidisciplinary design process to envision clinically relevant VLM interactions, and co-designed four VLM use concepts: Draft Report Generation, Augmented Report Review, Visual Search and Querying, and Patient Imaging History Highlights. We studied these concepts with 13 radiologists and clinicians who assessed the VLM concepts as valuable, yet articulated many design considerations. Reflecting on our findings, we discuss implications for integrating VLM capabilities in radiology, and for healthcare AI more generally.

Key facts. Cited by 45 papers in the map (data through 7 Oct 2026). First posted on arXiv 22 Feb 2024.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 14; 2025: 23; 2026: 8.

45 papers.

Sort:

By area:

  1. 2024MAIRA-2: Grounded Radiology Report Generationcited by 140
  2. 2025A Survey of State of the Art Large Vision Language Models: Alignment, Benchmark, Evaluations and Challengescited by 46
  3. 2024PadChest-GR: A Bilingual Chest X-ray Dataset for Grounded Radiology Report Generationcited by 35
  4. 2024An X-Ray Is Worth 15 Features: Sparse Autoencoders for Interpretable Radiology Report Generationcited by 23
  5. 2025Comprehensive language-image pre-training for 3D medical image understandingcited by 20
  6. 2025Understanding the LLM-ification of CHI: Unpacking the Impact of LLMs at CHI through a Systematic Literature Reviewcited by 16
  7. 2025AI, Help Me Thinkunicodex2014but for Myself: Assisting People in Complex Decision-Making by Providing Different Kinds of Cognitive Supportcited by 12
  8. 2024FactCheXcker: Mitigating Measurement Hallucinations in Chest X-ray Report Generation Modelscited by 11
  9. 2024Enhancing Human-Computer Interaction in Chest X-ray Analysis using Vision and Language Model with Eye Gaze Patternscited by 8
  10. 2024From Text to Multimodality: Exploring the Evolution and Impact of Large Language Models in Medical Practicecited by 8
  11. 2025A Multimodal Multi-Agent Framework for Radiology Report Generationcited by 8
  12. 2025EchoBench: Benchmarking Sycophancy in Medical Large Vision-Language Modelscited by 8
33 more
  1. 2024Visual Prompt Engineering for Vision Language Models in Radiologycited by 6
  2. 2025RadImageNet-VQA: A Large-Scale CT and MRI Dataset for Radiologic Visual Question Answeringcited by 6
  3. 2024NL-Eye: Abductive NLI for Imagescited by 5
  4. 2025CCD: Mitigating Hallucinations in Radiology MLLMs via Clinical Contrastive Decodingcited by 4
  5. 2024A Reliable Framework for Human-in-the-Loop Anomaly Detection in Time Seriescited by 3
  6. 2024Survey of User Interface Design and Interaction Techniques in Generative AI Applicationscited by 3
  7. 2024AI-Enhanced Sensemaking: Exploring the Design of a Generative AI-Based Assistant to Support Genetic Professionalscited by 3
  8. 2024Keypoint-Integrated Instruction-Following Data Generation for Enhanced Human Pose and Action Understanding in Multimodal Modelscited by 2
  9. 2025Augmenting Human Cognition With Generative AI: Lessons From AI-Assisted Decision-Makingcited by 2
  10. 2025Prompting without Panic: Attribute-aware, Zero-shot, Test-Time Calibrationcited by 2
  11. 2025Insights into a radiology-specialised multimodal large language model with sparse autoencoderscited by 2
  12. 2025Are Neuro-Inspired Multi-Modal Vision-Language Models Resilient to Membership Inference Privacy Leakage?cited by 2
  13. 2024"It depends": Configuring AI to Improve Clinical Usefulness Across Contextscited by 1
  14. 2025Your voice is your voice: Supporting Self-expression through Speech Generation and LLMs in Augmented and Alternative Communicationcited by 1
  15. 2025Adaptive Human-Agent Teaming: A Review of Empirical Studies from the Process Dynamics Perspectivecited by 1
  16. 2025Position: Restructuring of Categories and Implementation of Guidelines Essential for VLM Adoption in Healthcarecited by 1
  17. 2025LLaVA-Pose: Enhancing Human Pose and Action Understanding via Keypoint-Integrated Instruction Tuningcited by 1
  18. 2025FairSSL: Fair Multimodal Self-Supervised Learningcited by 1
  19. 2025Generalised Medical Phrase Groundingcited by 1
  20. 2026Augmenting Clinical Decision-Making with an Interactive and Interpretable AI Copilot: A Real-World User Study with Clinicians in Nephrology and Obstetricscited by 1
  21. 2026RDFace: A Benchmark Dataset for Rare Disease Facial Image Analysis under Extreme Data Scarcity and Phenotype-Aware Synthetic Generationcited by 1
  22. 2026Sycophancy Undermines Epistemic Vigilance in Cooperative Vision-Language Taskscited by 1
  23. 2026RadMatch: Auditable Radiology Report Evaluation via Finding-Level Matchingcited by 1
  24. 2024Towards Child-Inclusive Clinical Video Understanding for Autism Spectrum Disordercited by 0
  25. 2025VICCA: Visual Interpretation and Comprehension of Chest X-ray Anomalies in Generated Report Without Human Feedbackcited by 0
  26. 2025Aligning Proteins and Language: A Foundation Model for Protein Retrievalcited by 0
  27. 2025A Human-Centered Approach to Identifying Promises, Risks, & Challenges of Text-to-Image Generative AI in Radiologycited by 0
  28. 2025Closing the Performance Gap Between AI and Radiologists in Chest X-Ray Reportingcited by 0
  29. 2025Radiology Report Generation with Layer-Wise Anatomical Attentioncited by 0
  30. 2026Within the MDT Room: Situated in Multidisciplinary Team-Grounded Agent Debate for Clinical Diagnosiscited by 0
  31. 2026A Cloud-Edge System for Multimodal Clinical Screening in Resource-Constrained Rural Settingscited by 0
  32. 2026Toward Vision Language Model-based Assessment of Clinical Quality and Usability of LGE-MR Images for Cardiac Ablation Planningcited by 0
  33. 2026M2G-LLM: Enhancing Clinical Prediction via Multimodal Graph Reasoning and LLM Context Injectioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Toward a Vision-Language Foundation Model for Medical Data: Multimodal Dataset and Benchmarks for Vietnamese PET/CT Report Generation
  2. 2025Vision Language Models in Medicine
  3. 2023Vision-Language Modelling For Radiological Imaging and Reports In The Low Data Regime
  4. 2024Expert-level vision-language foundation model for real-world radiology and comprehensive evaluation
  5. 2025Evaluating Vision Language Model Adaptations for Radiology Report Generation in Low-Resource Languages
  6. 2025A Reality Check of Vision-Language Pre-training in Radiology: Have We Progressed Using Text?
  7. 2023RaDialog: A Large Vision-Language Model for Radiology Report Generation and Conversational Assistance
  8. 2025RadVLM: A Multitask Conversational Vision-Language Model for Radiology