NLG Evaluation: Past, Present, Future
Not on the map. Why?
Abstract (arXiv)
Natural Language Generation (NLG) evaluation has changed dramatically since 1990, and will continue to evolve in the future. In 1990, when NLG had close ties to linguistics, there was very little formal experimental evaluation in the modern sense. In 2026, when NLG is closely linked to machine learning, experimental evaluation is expected and indeed fundamental to research. Many evaluation techniques were developed over this period, including most recently LLM-as-Judge. I expect NLG evaluation will continue to evolve in the future. In particular, impact, qualitative, and safety evaluation will become more important as large numbers of people routinely use NLG technology.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 9 of its 41 references that are in the map. First posted on arXiv 22 May 2026.
Builds on
9 of 41 references.
References · 9
- 2020Beyond Accuracy: Behavioral Testing of NLP models with CheckListcited by 1,006
- 2024Leak, Cheat, Repeat: Data Contamination and Evaluation Malpractices in Closed-Source LLMscited by 229
- 2020Tangled up in BLEU: Reevaluating the Evaluation of Automatic Machine Translation Evaluation Metricscited by 158
- 2025Measuring what Matters: Construct Validity in Large Language Model Benchmarkscited by 119
- 2026MIRAGE: The Illusion of Visual Understandingcited by 65
- 2020Evaluating Semantic Accuracy of Data-to-Text Generation with Natural Language Inferencecited by 43
- 2020A Gold Standard Methodology for Evaluating Accuracy in Data-To-Text Systemscited by 34
- 2022Deconstructing NLG Evaluation: Evaluation Practices, Assumptions, and Their Implicationscited by 32
- 2025We Should Evaluate Real-World Impactcited by 12
Built on by
Citing papers per year: 2026: 2.
2 papers.