# Can LLMs Correct Physicians, Yet? Investigating Effective Interaction Methods in the Medical Domain

arXiv: https://arxiv.org/abs/2403.20288 · HTML: https://mlsanity.com/paper/2403.20288/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 24 references that are in the map.
- First posted on arXiv 29 Mar 2024.

## Builds on (most cited first, 12 of 12)

- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Exams](https://mlsanity.com/paper/2009.13081/) (2020)
- [PubMedQA: A Dataset for Biomedical Research Question Answering](https://mlsanity.com/paper/1909.06146/) (2019)
- [Capabilities of GPT-4 on Medical Challenge Problems](https://mlsanity.com/paper/2303.13375/) (2023)
- [Can Generalist Foundation Models Outcompete Special-Purpose Tuning? Case Study in Medicine](https://mlsanity.com/paper/2311.16452/) (2023)
- [MedAgents: Large Language Models as Collaborators for Zero-shot Medical Reasoning](https://mlsanity.com/paper/2311.10537/) (2023)
- [Can large language models reason about medical questions?](https://mlsanity.com/paper/2207.08143/) (2022)
- [Empowering Psychotherapy with Large Language Models: Cognitive Distortion Detection through Diagnosis of Thought Prompting](https://mlsanity.com/paper/2310.07146/) (2023)
- [Few shot chain-of-thought driven reasoning to prompt LLMs for open ended medical question answering](https://mlsanity.com/paper/2403.04890/) (2024)
- [Analogy Generation by Prompting Large Language Models: A Case Study of InstructGPT](https://mlsanity.com/paper/2210.04186/) (2022)
- [The Magic of IF: Investigating Causal Reasoning Abilities in Large Language Models of Code](https://mlsanity.com/paper/2305.19213/) (2023)
- [ChatGPT-HealthPrompt. Harnessing the Power of XAI in Prompt-Based Healthcare Decision Support using ChatGPT](https://mlsanity.com/paper/2308.09731/) (2023)

## Built on by (most cited first, 7 of 7)

- [Fairness Testing of Large Language Models in Role-Playing](https://mlsanity.com/paper/2411.00585/) (2024)
- [Gradient Boosting Decision Trees on Medical Diagnosis over Tabular Data](https://mlsanity.com/paper/2410.03705/) (2024)
- [XAI4LLM. Let Machine Learning Models and LLMs Collaborate for Enhanced In-Context Learning in Healthcare](https://mlsanity.com/paper/2405.06270/) (2024)
- [The Epistemological Consequences of Large Language Models: Rethinking collective intelligence and institutional knowledge](https://mlsanity.com/paper/2512.19570/) (2025)
- [Language Models And A Second Opinion Use Case: The Pocket Professional](https://mlsanity.com/paper/2410.20636/) (2024)
- [Zoom In Disparities in Healthcare LLM Q&A](https://mlsanity.com/paper/2510.17476/) (2025)
- [Investigating LLM Capabilities on Long Context Comprehension for Medical Question Answering](https://mlsanity.com/paper/2510.18691/) (2025)

## Abstract (arXiv)

We explore the potential of Large Language Models (LLMs) to assist and potentially correct physicians in medical decision-making tasks. We evaluate several LLMs, including Meditron, Llama2, and Mistral, to analyze the ability of these models to interact effectively with physicians across different scenarios. We consider questions from PubMedQA and several tasks, ranging from binary (yes/no) responses to long answer generation, where the answer of the model is produced after an interaction with a physician. Our findings suggest that prompt design significantly influences the downstream accuracy of LLMs and that LLMs can provide valuable feedback to physicians, challenging incorrect diagnoses and contributing to more accurate decision-making. For example, when the physician is accurate 38% of the time, Mistral can produce the correct answer, improving accuracy up to 74% depending on the prompt being used, while Llama2 and Meditron models exhibit greater sensitivity to prompt choice. Our analysis also uncovers the challenges of ensuring that LLM-generated suggestions are pertinent and useful, emphasizing the need for further research in this area.