mlsanityData to 7 Oct 2026

Evaluation of Large Language Models for Summarization Tasks in the Medical Domain: A Narrative Review

arXiv 2409.18170 ↗First posted 26 Sep 2024cs.CL cs.AIcited by 3

Not on the map. Why?

Abstract (arXiv)

Large Language Models have advanced clinical Natural Language Generation, creating opportunities to manage the volume of medical text. However, the high-stakes nature of medicine requires reliable evaluation, which remains a challenge. In this narrative review, we assess the current evaluation state for clinical summarization tasks and propose future directions to address the resource constraints of expert human evaluation.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 62 of its 110 references that are in the map. First posted on arXiv 26 Sep 2024.

Builds on

62 of 110 references.

References · 62

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2017Proximal Policy Optimization Algorithmscited by 14,441
  4. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  5. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  6. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  7. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  8. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  9. 2017Deep reinforcement learning from human preferencescited by 3,846
  10. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  11. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
  12. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,175
38 more
  1. 2023A Survey of Large Language Modelscited by 2,537
  2. 2019Fine-Tuning Language Models from Human Preferencescited by 2,124
  3. 2014CIDEr: Consensus-based Image Description Evaluationcited by 2,027
  4. 2022Large Language Models Encode Clinical Knowledgecited by 1,807
  5. 2020BLEURT: Learning Robust Metrics for Text Generationcited by 1,196
  6. 2020COMET: A Neural Framework for MT Evaluationcited by 1,024
  7. 2020Domain-Specific Language Model Pretraining for Biomedical Natural Language Processingcited by 996
  8. 2019Publicly Available Clinical BERT Embeddingscited by 995
  9. 2024SimPO: Simple Preference Optimization with a Reference-Free Rewardcited by 951
  10. 2023A General Theoretical Paradigm to Understand Learning from Human Preferencescited by 906
  11. 2016SPICE: Semantic Propositional Image Caption Evaluationcited by 820
  12. 2021BARTScore: Evaluating Generated Text as Text Generationcited by 719
  13. 2024ORPO: Monolithic Preference Optimization without Reference Modelcited by 556
  14. 2019MoverScore: Text Generation Evaluating with Contextualized Embeddings and Earth Mover Distancecited by 415
  15. 2024Contrastive Preference Optimization: Pushing the Boundaries of LLM Performance in Machine Translationcited by 390
  16. 2023Med-Flamingo: a Multimodal Medical Few-shot Learnercited by 387
  17. 2023Statistical Rejection Sampling Improves Preference Optimizationcited by 321
  18. 2024Iterative Reasoning Preference Optimizationcited by 244
  19. 2020Self-Alignment Pretraining for Biomedical Entity Representationscited by 233
  20. 2024Self-Play Preference Optimization for Language Model Alignmentcited by 228
  21. 2016Enhanced LSTM for Natural Language Inferencecited by 199
  22. 2024Direct Nash Optimization: Teaching Language Models to Self-Improve with General Preferencescited by 176
  23. 2020A Survey of Evaluation Metrics Used for NLG Systemscited by 164
  24. 2023Med-HALT: Medical Domain Hallucination Test for Large Language Modelscited by 156
  25. 2023Benchmarking Cognitive Biases in Large Language Models as Evaluatorscited by 147
  26. 2024Provably Robust DPO: Aligning Language Models with Noisy Feedbackcited by 133
  27. 2024DPO Meets PPO: Reinforced Token Optimization for RLHFcited by 126
  28. 2024Language Models Learn to Mislead Humans via RLHFcited by 118
  29. 2024Noise Contrastive Alignment of Language Models with Explicit Rewardscited by 85
  30. 2019Beyond BLEU: Training Neural Machine Translation with Semantic Similaritycited by 84
  31. 2024Learn Your Reference Model for Real Good Alignmentcited by 56
  32. 2018Towards a Better Metric for Evaluating Question Generation Systemscited by 52
  33. 2020Generating (Factual?) Narrative Summaries of RCTs: Experiments with Neural Multi-Document Summarizationcited by 52
  34. 2020Automated Lay Language Summarization of Biomedical Scientific Reviewscited by 46
  35. 2024Binary Classifier Optimization for Large Language Model Alignmentcited by 44
  36. 2024Towards Efficient Exact Optimization of Language Model Alignmentcited by 37
  37. 2020NUBIA: NeUral Based Interchangeability Assessor for Text Generationcited by 34
  38. 2022Human Evaluation and Correlation with Automatic Metrics in Consultation Note Generationcited by 33

50 most cited of 62 All 62 (JSON)

Built on by

Cited by year

Citing papers per year: 2025: 3.

3 papers.

  1. 2025Multi-Agent-as-Judge: Aligning LLM-Agent-Based Automated Evaluation with Multi-Dimensional Human Evaluationcited by 26
  2. 2025CounselBench: A Large-Scale Expert Evaluation and Adversarial Benchmarking of Large Language Models in Mental Health Question Answeringcited by 20
  3. 2025When AIs Judge AIs: The Rise of Agent-as-a-Judge Evaluation for LLMscited by 16