mlsanityData to 7 Oct 2026

DocLens: Multi-aspect Fine-grained Evaluation for Medical Text Generation

arXiv 2311.09581 ↗First posted 16 Nov 2023cs.CLcited by 16

Not on the map. Why?

Abstract (arXiv)

Medical text generation aims to assist with administrative work and highlight salient information to support decision-making. To reflect the specific requirements of medical text, in this paper, we propose a set of metrics to evaluate the completeness, conciseness, and attribution of the generated text at a fine-grained level. The metrics can be computed by various types of evaluators including instruction-following (both proprietary and open-source) and supervised entailment models. We demonstrate the effectiveness of the resulting framework, DocLens, with three evaluators on three tasks: clinical note generation, radiology report summarization, and patient question summarization. A comprehensive human study shows that DocLens exhibits substantially higher agreement with the judgments of medical experts than existing metrics. The results also highlight the need to improve open-source evaluators and suggest potential directions.

Key facts. Cited by 16 papers in the map (data through 7 Oct 2026). Builds on 30 of its 51 references that are in the map. First posted on arXiv 16 Nov 2023.

Builds on

30 of 51 references.

References · 30

  1. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  2. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,205
  3. 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,362
  4. 2020What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Examscited by 1,324
  5. 2019PubMedQA: A Dataset for Biomedical Research Question Answeringcited by 1,271
  6. 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,259
  7. 2020BLEURT: Learning Robust Metrics for Text Generationcited by 1,196
  8. 2019Adversarial NLI: A New Benchmark for Natural Language Understandingcited by 798
  9. 2023Enabling Large Language Models to Generate Text with Citationscited by 584
  10. 2019SAMSum Corpus: A Human-annotated Dialogue Dataset for Abstractive Summarizationcited by 530
  11. 2020PathVQA: 30000+ Questions for Medical Visual Question Answeringcited by 467
  12. 2017"Liar, Liar Pants on Fire": A New Benchmark Dataset for Fake News Detectioncited by 361
18 more
  1. 2021RadGraph: Extracting Clinical Entities and Relations from Radiology Reportscited by 310
  2. 2023Evaluating Verifiability in Generative Search Enginescited by 297
  3. 2022RARR: Researching and Revising What Language Models Say, Using Language Modelscited by 286
  4. 2022Towards a Unified Multi-Dimensional Evaluator for Text Generationcited by 264
  5. 2019MultiFC: A Real-World Multi-Domain Dataset for Evidence-Based Fact Checking of Claimscited by 169
  6. 2018Lessons from Natural Language Inference in the Clinical Domaincited by 149
  7. 2020Improving Factual Completeness and Consistency of Image-to-Text Radiology Report Generationcited by 139
  8. 2023WiCE: Real-World Entailment for Claims in Wikipediacited by 118
  9. 2023ACI-BENCH: a Novel Ambient Clinical Intelligence Dataset for Benchmarking Automatic Visit Note Generationcited by 116
  10. 2019Optimizing the Factual Correctness of a Summary: A Study of Summarizing Radiology Reportscited by 96
  11. 2021Medical Visual Question Answering: A Surveycited by 93
  12. 2022BioBART: Pretraining and Evaluation of A Biomedical Generative Language Modelcited by 83
  13. 2022Improving the Factual Correctness of Radiology Report Generation with Semantic Rewardscited by 82
  14. 2022Generating Literal and Implied Subquestions to Fact-check Complex Claimscited by 66
  15. 2022Generating Scientific Claims for Zero-Shot Scientific Fact Checkingcited by 59
  16. 2021Leveraging Pretrained Models for Automatic Summarization of Doctor-Patient Conversationscited by 42
  17. 2021What's in a Summary? Laying the Groundwork for Advances in Hospital-Course Summarizationcited by 35
  18. 2023Extrinsically-Focused Evaluation of Omissions in Medical Summarizationcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 8; 2026: 5.

16 papers.

Sort:

By area:

  1. 2024MAIRA-2: Grounded Radiology Report Generationcited by 140
  2. 2025X-Reasoner: Towards Generalizable Reasoning Across Modalities and Domainscited by 30
  3. 2024Improving Model Factuality with Fine-grained Critique-based Evaluatorcited by 15
  4. 2024Halu-J: Critique-Based Hallucination Judgecited by 14
  5. 2025From Feedback to Checklists: Grounded Evaluation of AI-Generated Clinical Notescited by 4
  6. 2025MedVAL: Toward Expert-Level Medical Text Validation with Language Modelscited by 3
  7. 2025Optimizing Long-Form Clinical Text Generation with Claim-Based Rewardscited by 3
  8. 2025Interpreting LLM-as-a-Judge Policies via Verifiable Global Explanationscited by 2
  9. 2025Generate, Evaluate, Iterate: Synthetic Data for Human-in-the-Loop Refinement of LLM Judgescited by 2
  10. 2026When KV Cache Reuse Fails in Multi-Agent Systems: Cross-Candidate Interaction is Crucial for LLM Judgescited by 2
  11. 2025GEMA-Score: Granular Explainable Multi-Agent Scoring Framework for Radiology Report Evaluationcited by 1
  12. 2025FocusMed: A Large Language Model-based Framework for Enhancing Medical Question Summarization with Focus Identificationcited by 0
4 more
  1. 2026Almost Clinical: Linguistic properties of synthetic electronic health recordscited by 0
  2. 2026MAPLE: Medical Aspect-Based Summarization with Phrase-Level Evidencecited by 0
  3. 2026eTracer: Polarity-Aware Evidence Groundingcited by 0
  4. 2026Severity-Aware Curriculum Learning with Multi-Model Response Selection for Medical Text Generationcited by 0