mlsanityData to 14 Oct 2026

ROUGE 2.0: Updated and Improved Measures for Evaluation of Summarization Tasks

arXiv 1803.01937 ↗First posted 5 Mar 2018cs.IR cs.AI cs.CLcited by 67

Not on the map. Why?

Abstract (arXiv)

Evaluation of summarization tasks is extremely crucial to determining the quality of machine generated summaries. Over the last decade, ROUGE has become the standard automatic evaluation measure for evaluating summarization tasks. While ROUGE has been shown to be effective in capturing n-gram overlap between system and human composed summaries, there are several limitations with the existing ROUGE measures in terms of capturing synonymous concepts and coverage of topics. Thus, often times ROUGE scores do not reflect the true quality of summaries and prevents multi-faceted evaluation of summaries (i.e. by topics, by overall content coverage and etc). In this paper, we introduce ROUGE 2.0, which has several updated measures of ROUGE: ROUGE-N+Synonyms, ROUGE-Topic, ROUGE-Topic+Synonyms, ROUGE-TopicUniq and ROUGE-TopicUniq+Synonyms; all of which are improvements over the core ROUGE measures.

Key facts. Cited by 67 papers in the map (data through 14 Oct 2026). First posted on arXiv 5 Mar 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 5; 2022: 9; 2023: 10; 2024: 15; 2025: 16; 2026: 12.

67 papers.

Sort:

By area:

  1. 2024LayerSkip: Enabling Early Exit Inference and Self-Speculative Decodingcited by 307
  2. 2023Detecting and Preventing Hallucinations in Large Vision Language Modelscited by 256
  3. 2023Vision Language Models in Autonomous Driving: A Survey and Outlookcited by 126
  4. 2023Confronting Reward Model Overoptimization with Constrained RLHFcited by 96
  5. 2022An Empirical Survey on Long Document Summarization: Datasets, Models and Metricscited by 74
  6. 2023Advancing Transformer Architecture in Long-Context Large Language Models: A Comprehensive Surveycited by 72
  7. 2024Lynx: An Open Source Hallucination Evaluation Modelcited by 57
  8. 2021Automatic Text Evaluation through the Lens of Wasserstein Barycenterscited by 38
  9. 2021InfoLM: A New Metric to Evaluate Summarization & Data2Text Generationcited by 33
  10. 2022I Can't Believe There's No Images! Learning Visual Tasks Using only Language Supervisioncited by 26
  11. 2022How Far are We from Robust Long Abstractive Summarization?cited by 25
  12. 2024ProteinGPT: Multimodal LLM for Protein Property Prediction and Structure Understandingcited by 23
38 more
  1. 2023OpenViVQA: Task, Dataset, and Multimodal Fusion Models for Visual Question Answering in Vietnamesecited by 20
  2. 2024OLAPH: Improving Factuality in Biomedical Long-form Question Answeringcited by 20
  3. 2022A global analysis of metrics used for measuring performance in natural language processingcited by 17
  4. 2024Can We Verify Step by Step for Incorrect Answer Detection?cited by 15
  5. 2023Scaling Laws Do Not Scalecited by 13
  6. 2024RakutenAI-7B: Extending Large Language Models for Japanesecited by 13
  7. 2024A Comparative Study of Quality Evaluation Methods for Text Summarizationcited by 13
  8. 2025A Contemporary Survey of Large Language Model Assisted Program Analysiscited by 11
  9. 2021Fairness for Whom? Understanding the Reader's Perception of Fairness in Text Summarizationcited by 8
  10. 2024CRScore: Grounding Automated Evaluation of Code Review Comments in Code Claims and Smellscited by 8
  11. 2024Can LLMs Help Create Grammar?: Automating Grammar Creation for Endangered Languages with In-Context Learningcited by 8
  12. 2025Robust Change Captioning in Remote Sensing: SECOND-CC Dataset and MModalCC Frameworkcited by 8
  13. 2025DEL: Context-Aware Dynamic Exit Layer for Efficient Self-Speculative Decodingcited by 8
  14. 2025CML-Bench: A Framework for Evaluating and Enhancing LLM-Powered Movie Scripts Generationcited by 7
  15. 2026Self-Refining Vision Language Model for Robotic Failure Detection and Reasoningcited by 7
  16. 2026MAU-GPT: Enhancing Multi-type Industrial Anomaly Understanding via Anomaly-aware and Generalist Experts Adaptationcited by 6
  17. 2026Can AI Be a Good Peer Reviewer? A Survey of Peer Review Process, Evaluation, and the Futurecited by 6
  18. 2021To Point or Not to Point: Understanding How Abstractive Summarizers Paraphrase Textcited by 5
  19. 2023GPT Self-Supervision for a Better Data Annotatorcited by 5
  20. 2023ViCLEVR: A Visual Reasoning Dataset and Hybrid Multimodal Fusion Model for Visual Question Answering in Vietnamesecited by 5
  21. 2024Weighted Grouped Query Attention in Transformerscited by 5
  22. 2025A Comprehensive Graph Framework for Question Answering with Mode-Seeking Preference Alignmentcited by 5
  23. 2023A comprehensive review of automatic text summarization techniques: method, data, evaluation and codingcited by 4
  24. 2024The Right Time Matters: Data Arrangement Affects Zero-Shot Generalization in Instruction Tuningcited by 4
  25. 2025WILDCHAT-50M: A Deep Dive Into the Role of Synthetic Data in Post-Trainingcited by 4
  26. 2025ChronoLLM: Customizing Language Models for Physics-Based Simulation Code Generationcited by 3
  27. 2026ConfLayers: Adaptive Confidence-based Layer Skipping for Self-Speculative Decodingcited by 3
  28. 2021Improving Online Forums Summarization via Hierarchical Unified Deep Neural Networkcited by 2
  29. 2022WIDAR -- Weighted Input Document Augmented ROUGEcited by 2
  30. 2022ESSumm: Extractive Speech Summarization from Untranscribed Meetingcited by 2
  31. 2025eC-Tab2Text: Aspect-Based Text Generation from e-Commerce Product Tablescited by 2
  32. 2025Metric Design != Metric Behavior: Improving Metric Selection for the Unbiased Evaluation of Dimensionality Reductioncited by 2
  33. 2022Introducing the Welsh Text Summarisation Dataset and Baseline Systemscited by 1
  34. 2023Redundancy Aware Multi-Reference Based Gainwise Evaluation of Extractive Summarizationcited by 1
  35. 2024Refining Joint Text and Source Code Embeddings for Retrieval Task with Parameter-Efficient Fine-Tuningcited by 1
  36. 2024LLMs-in-the-loop Part-1: Expert Small AI Models for Bio-Medical Text Translationcited by 1
  37. 2024SparrowVQE: Visual Question Explanation for Course Content Understandingcited by 1
  38. 2024Can MLLMs generate human-like feedback in grading multimodal short answers?cited by 1

50 most cited of 67 All 67 (JSON)