mlsanityData to 7 Oct 2026

Evaluation of Text Generation: A Survey

arXiv 2006.14799 ↗First posted 26 Jun 2020cs.CL cs.LGcited by 275

Not on the map. Why?

Abstract (arXiv)

The paper surveys evaluation methods of natural language generation (NLG) systems that have been developed in the last few years. We group NLG evaluation methods into three categories: (1) human-centric evaluation metrics, (2) automatic metrics that require no training, and (3) machine-learned metrics. For each category, we discuss the progress that has been made and the challenges still being faced, with a focus on the evaluation of recently proposed NLG tasks and neural NLG models. We then present two examples for task-specific NLG evaluations for automatic text summarization and long text generation, and conclude the paper by proposing future research directions.

Key facts. Cited by 275 papers in the map (data through 7 Oct 2026). First posted on arXiv 26 Jun 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 41; 2022: 49; 2023: 58; 2024: 62; 2025: 32; 2026: 33.

275 papers.

Sort:

By area:

  1. 2023ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debatecited by 724
  2. 2021MAUVE: Measuring the Gap Between Neural Text and Human Text using Divergence Frontierscited by 466
  3. 2023Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defensecited by 401
  4. 2022CoAuthor: Designing a Human-AI Collaborative Writing Dataset for Exploring Language Model Capabilitiescited by 245
  5. 2021The GEM Benchmark: Natural Language Generation, its Evaluation and Metricscited by 239
  6. 2023Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificitycited by 204
  7. 2022Co-Writing Screenplays and Theatre Scripts with Language Models: An Evaluation by Industry Professionalscited by 191
  8. 2022ROSCOE: A Suite of Metrics for Scoring Step-by-Step Reasoningcited by 175
  9. 2021Hurdles to Progress in Long-form Question Answeringcited by 166
  10. 2022"I'm sorry to hear that": Finding New Biases in Language Models with a Holistic Descriptor Datasetcited by 160
  11. 2023A Study of Generative Large Language Model for Medical Research and Healthcarecited by 160
  12. 2022Repairing the Cracked Foundation: A Survey of Obstacles in Evaluation Practices for Generated Textcited by 154
38 more
  1. 2021Q2: Evaluating Factual Consistency in Knowledge-Grounded Dialogues via Question Generation and Question Answeringcited by 131
  2. 2022A Survey of Controllable Text Generation using Transformer-based Pre-trained Language Modelscited by 128
  3. 2023Similarity of Neural Network Models: A Survey of Functional and Representational Measurescited by 128
  4. 2021How much do language models copy from their training data? Evaluating linguistic novelty in text generation using RAVENcited by 124
  5. 2022Pretrained Language Models for Text Generation: A Surveycited by 121
  6. 2021Advances and Challenges in Conversational Recommender Systems: A Surveycited by 118
  7. 2023A Critical Evaluation of Evaluations for Long-form Question Answeringcited by 111
  8. 2023WebGLM: Towards An Efficient Web-Enhanced Question Answering System with Human Preferencescited by 100
  9. 2021The Perils of Using Mechanical Turk to Evaluate Open-Ended Text Generationcited by 96
  10. 2023LongEval: Guidelines for Human Evaluation of Faithfulness in Long-form Summarizationcited by 96
  11. 2021Learning Compact Metrics for MTcited by 93
  12. 2021QA Dataset Explosion: A Taxonomy of NLP Resources for Question Answering and Reading Comprehensioncited by 91
  13. 2023Pearl: Personalizing Large Language Model Writing Assistants with Generation-Calibrated Retrieverscited by 91
  14. 2021Is GPT-3 Text Indistinguishable from Human Text? Scarecrow: A Framework for Scrutinizing Machine Textcited by 89
  15. 2021Pre-trained Language Models in Biomedical Domain: A Systematic Surveycited by 87
  16. 2023Language Model Self-improvement by Reinforcement Learning Contemplationcited by 75
  17. 2024Natural Language Understanding and Inference with MLLM in Visual Question Answering: A Surveycited by 74
  18. 2022RankGen: Improving Text Generation with Large Ranking Modelscited by 72
  19. 2024LLM-based NLG Evaluation: Current Status and Challengescited by 71
  20. 2022CREPE: Open-Domain Question Answering with False Presuppositionscited by 70
  21. 2024LLM Reasoners: New Evaluation, Library, and Analysis of Step-by-Step Reasoning with Large Language Modelscited by 70
  22. 2023LLM Comparative Assessment: Zero-shot NLG Evaluation through Pairwise Comparisons using Large Language Modelscited by 66
  23. 2023CritiqueLLM: Towards an Informative Critique Generation Model for Evaluation of Large Language Model Generationcited by 65
  24. 2023Uncertainty in Natural Language Generation: From Theory to Applicationscited by 61
  25. 2025No Free Labels: Limitations of LLM-as-a-Judge Without Human Groundingcited by 50
  26. 2021Perturbation CheckLists for Evaluating NLG Evaluation Metricscited by 49
  27. 2022On the Blind Spots of Model-Based Evaluation Metrics for Text Generationcited by 48
  28. 2023EcomGPT: Instruction-tuning Large Language Models with Chain-of-Task Tasks for E-commercecited by 48
  29. 2023LongForm: Effective Instruction Tuning with Reverse Instructionscited by 47
  30. 2025SCORE: Story Coherence and Retrieval Enhancement for AI Narrativescited by 42
  31. 2022Neural Approaches to Conversational Information Retrievalcited by 41
  32. 2023ArguGPT: evaluating, understanding and identifying argumentative essays generated by GPT modelscited by 41
  33. 2023Evaluating Evaluation Metrics: A Framework for Analyzing NLG Evaluation Metrics using Measurement Theorycited by 41
  34. 2024When LLMs step into the 3D World: A Survey and Meta-Analysis of 3D Tasks via Multi-modal Large Language Modelscited by 41
  35. 2021FRUIT: Faithfully Reflecting Updated Information in Textcited by 39
  36. 2024Leveraging Large Language Models for NLG Evaluation: Advances and Challengescited by 37
  37. 2025Rating Roulette: Self-Inconsistency in LLM-As-A-Judge Frameworkscited by 37
  38. 2024Diff-eRank: A Novel Rank-Based Metric for Evaluating Large Language Modelscited by 35

50 most cited of 275 All 275 (JSON)