# VeriFact: Verifying Facts in LLM-Generated Clinical Text with Electronic Health Records

arXiv: https://arxiv.org/abs/2501.16672 · HTML: https://mlsanity.com/paper/2501.16672/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 0 of its 65 references that are in the map.
- First posted on arXiv 28 Jan 2025.

## Built on by (most cited first, 17 of 17)

- [Analyzing Uncertainty of LLM-as-a-Judge: Interval Evaluations with Conformal Prediction](https://mlsanity.com/paper/2509.18658/) (2025)
- [From Scores to Steps: Diagnosing and Improving LLM Performance in Evidence-Based Medical Calculations](https://mlsanity.com/paper/2509.16584/) (2025)
- [DischargeSim: A Simulation Benchmark for Educational Doctor-Patient Communication at Discharge](https://mlsanity.com/paper/2509.07188/) (2025)
- [MedReadCtrl: Personalizing medical text generation with readability-controlled instruction learning](https://mlsanity.com/paper/2507.07419/) (2025)
- [Inflated Excellence or True Performance? Rethinking Medical Diagnostic Benchmarks with Dynamic Evaluation](https://mlsanity.com/paper/2510.09275/) (2025)
- [MedFactEval and MedAgentBrief: A Framework and Workflow for Generating and Evaluating Factual Clinical Summaries](https://mlsanity.com/paper/2509.05878/) (2025)
- [MedVAL: Toward Expert-Level Medical Text Validation with Language Models](https://mlsanity.com/paper/2507.03152/) (2025)
- [Adoption and Use of LLMs at an Academic Medical Center](https://mlsanity.com/paper/2602.00074/) (2026)
- [Verification Mirage: Mapping the Reliability Boundary of Self-Verification in Medical VQA](https://mlsanity.com/paper/2605.10850/) (2026)
- [Conformal Prediction for Risk-Controlled Medical Entity Extraction Across Clinical Domains](https://mlsanity.com/paper/2603.00924/) (2026)
- [PanCanBench: A Comprehensive Benchmark for Evaluating Large Language Models in Pancreatic Oncology](https://mlsanity.com/paper/2603.01343/) (2026)
- [Active Multiple-Prediction-Powered Inference](https://mlsanity.com/paper/2605.08429/) (2026)
- [CMQCIC-Bench: A Chinese Benchmark for Evaluating Large Language Models in Medical Quality Control Indicator Calculation](https://mlsanity.com/paper/2502.11703/) (2025)
- [Performance Assessment Strategies for Language Model Applications in Healthcare](https://mlsanity.com/paper/2509.08087/) (2025)
- [Deployment-Centered Evaluation: Predicting Query-Level Rejection Risk in a Clinical LLM System](https://mlsanity.com/paper/2606.12702/) (2026)
- [Review Before Trust: Source-Grounded Integrity Gates for AI-Assisted Personal Health Records](https://mlsanity.com/paper/2608.29965/) (2026)
- [The Privacy-Hallucination Tradeoff in Differentially Private Language Models](https://mlsanity.com/paper/2609.00492/) (2026)

## Abstract (arXiv)

Methods to ensure factual accuracy of text generated by large language models (LLM) in clinical medicine are lacking. VeriFact is an artificial intelligence system that combines retrieval-augmented generation and LLM-as-a-Judge to verify whether LLM-generated text is factually supported by a patient's medical history based on their electronic health record (EHR). To evaluate this system, we introduce VeriFact-BHC, a new dataset that decomposes Brief Hospital Course narratives from discharge summaries into a set of simple statements with clinician annotations for whether each statement is supported by the patient's EHR clinical notes. Whereas highest agreement between clinicians was 88.5%, VeriFact achieves up to 92.7% agreement when compared to a denoised and adjudicated average human clinican ground truth, suggesting that VeriFact exceeds the average clinician's ability to fact-check text against a patient's medical record. VeriFact may accelerate the development of LLM-based EHR applications by removing current evaluation bottlenecks.