mlsanityData to 7 Oct 2026

Towards objectively evaluating the quality of generated medical summaries

arXiv 2104.04412 ↗First posted 9 Apr 2021cs.CLcited by 5

Not on the map. Why?

Abstract (arXiv)

We propose a method for evaluating the quality of generated text by asking evaluators to count facts, and computing precision, recall, f-score, and accuracy from the raw counts. We believe this approach leads to a more objective and easier to reproduce evaluation. We apply this to the task of medical report summarisation, where measuring objective quality and accuracy is of paramount importance.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 10 of its 20 references that are in the map. First posted on arXiv 9 Apr 2021.

Builds on

10 of 20 references.

References · 10

  1. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  2. 2019PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarizationcited by 1,100
  3. 2019Evaluating the Factual Consistency of Abstractive Text Summarizationcited by 578
  4. 2020Evaluation of Text Generation: A Surveycited by 275
  5. 2019Leveraging BERT for Extractive Text Summarization on Lecturescited by 63
  6. 2019Few-Shot NLG with Pre-Trained Language Modelcited by 52
  7. 2020Evaluating Semantic Accuracy of Data-to-Text Generation with Natural Language Inferencecited by 43
  8. 2020A Gold Standard Methodology for Evaluating Accuracy in Data-To-Text Systemscited by 34
  9. 2019ViGGO: A Video Game Corpus for Data-To-Text Generation in Open-Domain Conversationcited by 31
  10. 2018Neural Latent Extractive Document Summarizationcited by 20

Built on by

Cited by year

Citing papers per year: 2021: 1; 2022: 2; 2023: 2.

5 papers.

  1. 2022Summarizing Patients Problems from Hospital Progress Notes Using Pre-trained Sequence-to-Sequence Modelscited by 30
  2. 2022A Survey on Medical Document Summarizationcited by 14
  3. 2023A Meta-Evaluation of Faithfulness Metrics for Long-Form Hospital-Course Summarizationcited by 14
  4. 2023A Survey for Biomedical Text Summarization: From Pre-trained to Large Language Modelscited by 3
  5. 2021Conical Classification For Computationally Efficient One-Class Topic Determinationcited by 0