# Deciphering Diagnoses: How Large Language Models Explanations Influence Clinical Decision Making

arXiv: https://arxiv.org/abs/2310.01708 · HTML: https://mlsanity.com/paper/2310.01708/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 23 references that are in the map.
- First posted on arXiv 3 Oct 2023.

## Builds on (most cited first, 9 of 9)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Medical Image Captioning via Generative Pretrained Transformers](https://mlsanity.com/paper/2209.13983/) (2022)
- [Understanding the Effect of Counterfactual Explanations on Trust and Reliance on AI for Human-AI Collaborative Clinical Decision Making](https://mlsanity.com/paper/2308.04375/) (2023)
- [SummQA at MEDIQA-Chat 2023:In-Context Learning with GPT-4 for Medical Summarization](https://mlsanity.com/paper/2306.17384/) (2023)
- [RuMedBench: A Russian Medical Language Understanding Benchmark](https://mlsanity.com/paper/2201.06499/) (2022)
- [ExplainCPE: A Free-text Explanation Benchmark of Chinese Pharmacist Examination](https://mlsanity.com/paper/2305.12945/) (2023)
- [Predicting Clinical Diagnosis from Patients Electronic Health Records Using BERT-based Neural Networks](https://mlsanity.com/paper/2007.07562/) (2020)

## Built on by (most cited first, 2 of 2)

- [CLIN-LLM: A Safety-Constrained Hybrid Framework for Clinical Diagnosis and Treatment Generation](https://mlsanity.com/paper/2510.22609/) (2025)
- [CDR-Agent: Intelligent Selection and Execution of Clinical Decision Rules Using Large Language Model Agents](https://mlsanity.com/paper/2505.23055/) (2025)

## Abstract (arXiv)

Clinical Decision Support Systems (CDSS) utilize evidence-based knowledge and patient data to offer real-time recommendations, with Large Language Models (LLMs) emerging as a promising tool to generate plain-text explanations for medical decisions. This study explores the effectiveness and reliability of LLMs in generating explanations for diagnoses based on patient complaints. Three experienced doctors evaluated LLM-generated explanations of the connection between patient complaints and doctor and model-assigned diagnoses across several stages. Experimental results demonstrated that LLM explanations significantly increased doctors' agreement rates with given diagnoses and highlighted potential errors in LLM outputs, ranging from 5% to 30%. The study underscores the potential and challenges of LLMs in healthcare and emphasizes the need for careful integration and evaluation to ensure patient safety and optimal clinical utility.