# Beyond Multiple-Choice Accuracy: Real-World Challenges of Implementing Large Language Models in Healthcare

arXiv: https://arxiv.org/abs/2410.18460 · HTML: https://mlsanity.com/paper/2410.18460/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 30 references that are in the map.
- First posted on arXiv 24 Oct 2024.

## Builds on (most cited first, 14 of 14)

- [Can large language models reason about medical questions?](https://mlsanity.com/paper/2207.08143/) (2022)
- [Large language models in healthcare and medical domain: A review](https://mlsanity.com/paper/2401.06775/) (2023)
- [Towards Mitigating Hallucination in Large Language Models via Self-Reflection](https://mlsanity.com/paper/2310.06271/) (2023)
- [Towards Conversational Diagnostic AI](https://mlsanity.com/paper/2401.05654/) (2024)
- [Matching Patients to Clinical Trials with Large Language Models](https://mlsanity.com/paper/2307.15051/) (2023)
- [Towards Evaluating and Building Versatile Large Language Models for Medicine](https://mlsanity.com/paper/2408.12547/) (2024)
- [Elephants Never Forget: Memorization and Learning of Tabular Data in Large Language Models](https://mlsanity.com/paper/2404.06209/) (2024)
- [Unmasking and Quantifying Racial Bias of Large Language Models in Medical Report Generation](https://mlsanity.com/paper/2401.13867/) (2024)
- [Efficient Fine-Tuning of Large Language Models for Automated Medical Documentation](https://mlsanity.com/paper/2409.09324/) (2024)
- [The Reasonable Effectiveness of Diverse Evaluation Data](https://mlsanity.com/paper/2301.09406/) (2023)
- [Adversarial Attacks on Large Language Models in Medicine](https://mlsanity.com/paper/2406.12259/) (2024)
- [Prompt Injection Attacks on Large Language Models in Oncology](https://mlsanity.com/paper/2407.18981/) (2024)
- [Medical Foundation Models are Susceptible to Targeted Misinformation Attacks](https://mlsanity.com/paper/2309.17007/) (2023)
- [A survey of recent methods for addressing AI fairness and bias in biomedicine](https://mlsanity.com/paper/2402.08250/) (2024)

## Built on by (most cited first, 4 of 4)

- [Memorization in Large Language Models in Medicine: Prevalence, Characteristics, and Implications](https://mlsanity.com/paper/2509.08604/) (2025)
- [CLARITY: Clinical Assistant for Routing, Inference, and Triage](https://mlsanity.com/paper/2510.02463/) (2025)
- [How to Evaluate Medical AI](https://mlsanity.com/paper/2509.11941/) (2025)
- [A Field Guide to Deploying AI Agents in Clinical Practice](https://mlsanity.com/paper/2509.26153/) (2025)

## Abstract (arXiv)

Large Language Models (LLMs) have gained significant attention in the medical domain for their human-level capabilities, leading to increased efforts to explore their potential in various healthcare applications. However, despite such a promising future, there are multiple challenges and obstacles that remain for their real-world uses in practical settings. This work discusses key challenges for LLMs in medical applications from four unique aspects: operational vulnerabilities, ethical and social considerations, performance and assessment difficulties, and legal and regulatory compliance. Addressing these challenges is crucial for leveraging LLMs to their full potential and ensuring their responsible integration into healthcare.