# CareTransition-Audit: A Benchmark to Audit Discharge Summaries for Efficient Care Transitions

arXiv: https://arxiv.org/abs/2604.05435 · HTML: https://mlsanity.com/paper/2604.05435/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 40 references that are in the map.
- First posted on arXiv 7 Apr 2026.

## Builds on (most cited first, 12 of 12)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Large Language Models Encode Clinical Knowledge](https://mlsanity.com/paper/2212.13138/) (2022)
- [Phi-4 Technical Report](https://mlsanity.com/paper/2412.08905/) (2024)
- [RLAIF vs. RLHF: Scaling Reinforcement Learning from Human Feedback with AI Feedback](https://mlsanity.com/paper/2309.00267/) (2023)
- [Large language models in healthcare and medical domain: A review](https://mlsanity.com/paper/2401.06775/) (2023)
- [Med42-v2: A Suite of Clinical LLMs](https://mlsanity.com/paper/2408.06142/) (2024)
- [Overview of the First Shared Task on Clinical Text Generation: RRG24 and "Discharge Me!"](https://mlsanity.com/paper/2409.16603/) (2024)
- [Development and Validation of the Provider Documentation Summarization Quality Instrument for Large Language Models](https://mlsanity.com/paper/2501.08977/) (2025)
- [A Method to Automate the Discharge Summary Hospital Course for Neurology Patients](https://mlsanity.com/paper/2305.06416/) (2023)
- [e-Health CSIRO at "Discharge Me!" 2024: Generating Discharge Summary Sections with Fine-tuned Language Models](https://mlsanity.com/paper/2407.02723/) (2024)
- [Large Language Model-Based Generation of Discharge Summaries](https://mlsanity.com/paper/2512.06812/) (2025)

## Abstract (arXiv)

Incomplete or inconsistent discharge documentation drives care fragmentation and avoidable readmissions. Despite its critical role in patient safety, auditing discharge summaries relies on manual review and does not scale. We propose an automated framework for auditing discharge summaries using large language models (LLMs). Our approach operationalizes the DISCHARGED framework into a checklist of 46 questions. Using 50 summaries from the MIMIC-IV database, with clinician ground-truth labels, we benchmark 11 LLMs. Model-assessed mean documentation completeness ranges from 54.9% to 74.2%, and the best-performing models achieve a Cohen's kappa values around 0.5 against clinician labels, indicating moderate agreement. All models struggle to identify ambiguous documentation (Unclear), highlighting a key gap in current automated auditing. This work provides a clinician-validated benchmark and zero-shot baselines for systematic quality improvement in clinical documentation.