mlsanityData to 7 Oct 2026

Trustworthy and Practical AI for Healthcare: A Guided Deferral System with Large Language Models

arXiv 2406.07212 ↗First posted 11 Jun 2024cs.CL cs.AI cs.HCcited by 16

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) offer a valuable technology for various applications in healthcare. However, their tendency to hallucinate and the existing reliance on proprietary systems pose challenges in environments concerning critical decision-making and strict data privacy regulations, such as healthcare, where the trust in such systems is paramount. Through combining the strengths and discounting the weaknesses of humans and AI, the field of Human-AI Collaboration (HAIC) presents one front for tackling these challenges and hence improving trust. This paper presents a novel HAIC guided deferral system that can simultaneously parse medical reports for disorder classification, and defer uncertain predictions with intelligent guidance to humans. We develop methodology which builds efficient, effective and open-source LLMs for this purpose, for the real-world deployment in healthcare. We conduct a pilot study which showcases the effectiveness of our proposed system in practice. Additionally, we highlight drawbacks of standard calibration metrics in imbalanced data scenarios commonly found in healthcare, and suggest a simple yet effective solution: the Imbalanced Expected Calibration Error.

Key facts. Cited by 16 papers in the map (data through 7 Oct 2026). Builds on 20 of its 40 references that are in the map. First posted on arXiv 11 Jun 2024.

Builds on

20 of 40 references.

References · 20

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2017On Calibration of Modern Neural Networkscited by 4,860
  3. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  4. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
  5. 2019CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparisoncited by 1,410
  6. 2023Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMscited by 946
  7. 2023Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedbackcited by 788
  8. 2022Teaching Models to Express Their Uncertainty in Wordscited by 731
  9. 2023Instruction Tuning for Large Language Models: A Surveycited by 617
  10. 2019Measuring Calibration in Deep Learningcited by 443
  11. 2020Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performancecited by 396
  12. 2017Predict Responsibly: Improving Fairness and Accuracy by Learning to Defercited by 228
8 more
  1. 2022Out-of-Distribution Detection and Selective Generation for Conditional Language Modelscited by 193
  2. 2021Reframing Human-AI Collaboration for Generating Free-Text Explanationscited by 125
  3. 2020Mitigating Bias in Calibration Error Estimationcited by 100
  4. 2023Adaptation with Self-Evaluation to Improve Selective Prediction in LLMscited by 49
  5. 2023Supporting Human-AI Collaboration in Auditing LLMs with LLMscited by 46
  6. 2023Radiology-Llama2: Best-in-Class Large Language Model for Radiologycited by 42
  7. 2023Post-Abstention: Towards Reliably Re-Attempting the Abstained Instances in QAcited by 20
  8. 2024CheX-GPT: Harnessing Large Language Models for Enhanced Chest X-ray Report Labelingcited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 8; 2026: 8.

16 papers.

Sort:

By area:

  1. 2025Medical Reasoning in the Era of LLMs: A Systematic Review of Enhancement Techniques and Applicationscited by 10
  2. 2025TAMA: A Human-AI Collaborative Thematic Analysis Framework Using Multi-Agent LLMs for Clinical Interviewscited by 9
  3. 2025Adversarial Robustness in One-Stage Learning-to-Defercited by 5
  4. 2025Identity-Free Deferral For Unseen Expertscited by 4
  5. 2025Selective Prediction Reduces the Negative Effects of Automation Bias Overall but Increases False Negativescited by 2
  6. 2025Benchmarking Large Language Models for Diagnosing Students' Cognitive Skills from Handwritten Math Workcited by 1
  7. 2025From Deferral to Learning: Online In-Context Knowledge Distillation for LLM Cascadescited by 1
  8. 2026Consistent Learning-to-Defer with Expert-Conditional Advicecited by 1
  9. 2026Process Supervision of Confidence Margin for Calibrated LLM Reasoningcited by 1
  10. 2025Fail Fast, or Ask: Mitigating the Deficiencies of Reasoning LLMs with Human-in-the-Loop Systems Engineeringcited by 0
  11. 2026Multimodal Survival Analysis with Locally Deployable Large Language Modelscited by 0
  12. 2026Reliable Multilingual Orthopedic Decision Support from Clinical Narratives: Language-Aware Adaptation and Verification-Guided Deferralcited by 0
4 more
  1. 2026MedGuards: Multi-Agent System for Reliable Medical Error Detection and Correctioncited by 0
  2. 2026RACER: Role-Aligned Competence Estimation for Human-AI Routingcited by 0
  3. 2026Should I stay or should I show? Learning to selectively disclose informationcited by 0
  4. 2026OmniMed-Jev: Calibrating LVLM Confidence for Trustworthy Medical Multimodal Decisions via System Onecited by 0