mlsanityData to 14 Oct 2026

Reference-free Evaluation Metrics for Text Generation: A Survey

arXiv 2501.12011 ↗First posted 21 Jan 2025cs.CLcited by 9

Not on the map. Why?

Abstract (arXiv)

A number of automatic evaluation metrics have been proposed for natural language generation systems. The most common approach to automatic evaluation is the use of a reference-based metric that compares the model's output with gold-standard references written by humans. However, it is expensive to create such references, and for some tasks, such as response generation in dialogue, creating references is not a simple matter. Therefore, various reference-free metrics have been developed in recent years. In this survey, which intends to cover the full breadth of all NLG tasks, we investigate the most commonly used approaches, their application, and their other uses beyond evaluating models. The survey concludes by highlighting some promising directions for future research.

Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Builds on 74 of its 120 references that are in the map. First posted on arXiv 21 Jan 2025.

Builds on

74 of 120 references.

References · 74

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  4. 2020Scaling Laws for Neural Language Modelscited by 6,586
  5. 2017Deep reinforcement learning from human preferencescited by 3,912
  6. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,790
  7. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,682
  8. 2020Learning to summarize from human feedbackcited by 2,588
  9. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,424
  10. 2015A large annotated corpus for learning natural language inferencecited by 1,847
  11. 2015A Diversity-Promoting Objective Function for Neural Conversation Modelscited by 1,077
  12. 2018Neural Network Acceptability Judgmentscited by 958
38 more
  1. 2023Can Large Language Models Be an Alternative to Human Evaluations?cited by 734
  2. 2020SummEval: Re-evaluating Summarization Evaluationcited by 674
  3. 2018Annotation Artifacts in Natural Language Inference Datacited by 671
  4. 2023Large Language Models Are State-of-the-Art Evaluators of Translation Qualitycited by 420
  5. 2021A Survey of Data Augmentation Approaches for NLPcited by 409
  6. 2020Asking and Answering Questions to Evaluate the Factual Consistency of Summariescited by 378
  7. 2024Humans or LLMs as the Judge? A Study on Judgement Biasescited by 323
  8. 2021All That's 'Human' Is Not Gold: Evaluating Human Evaluation of Generated Textcited by 287
  9. 2020Evaluation of Text Generation: A Surveycited by 277
  10. 2023xCOMET: Transparent Machine Translation Evaluation through Fine-grained Error Detectioncited by 265
  11. 2021QuestEval: Summarization Asks for Fact-based Evaluationcited by 244
  12. 2017Why We Need New Evaluation Metrics for NLGcited by 233
  13. 2017Survey of the State of the Art in Natural Language Generation: Core tasks, applications and evaluationcited by 206
  14. 2020USR: An Unsupervised and Reference Free Evaluation Metric for Dialog Generationcited by 188
  15. 2020Reformulating Unsupervised Style Transfer as Paraphrase Generationcited by 183
  16. 2016A Hierarchical Latent Variable Encoder-Decoder Model for Generating Dialoguescited by 174
  17. 2020A Survey of Evaluation Metrics Used for NLG Systemscited by 164
  18. 2016An Actor-Critic Algorithm for Sequence Predictioncited by 159
  19. 2020Unsupervised Evaluation of Interactive Dialog with DialoGPTcited by 139
  20. 2020Automatic Machine Translation Evaluation in Many Languages via Zero-Shot Paraphrasingcited by 115
  21. 2020BLEU might be Guilty but References are not Innocentcited by 110
  22. 2019Answers Unite! Unsupervised Metrics for Reinforced Summarization Modelscited by 92
  23. 2019Comparison of Diverse Decoding Methods from Conditional Language Modelscited by 78
  24. 2019Mixture Models for Diverse Machine Translation: Tricks of the Tradecited by 77
  25. 2020GO FIGURE: A Meta Evaluation of Factuality in Summarizationcited by 76
  26. 2020SUPERT: Towards New Frontiers in Unsupervised Evaluation Metrics for Multi-Document Summarizationcited by 69
  27. 2022Fine-grained Image Captioning with CLIP Rewardcited by 69
  28. 2019Question Answering as an Automatic Evaluation Metric for News Article Summarizationcited by 63
  29. 2020UNION: An Unreferenced Metric for Evaluating Open-ended Story Generationcited by 60
  30. 2019Evaluating Coherence in Dialogue Systems using Entailmentcited by 53
  31. 2022On the Limitations of Reference-Free Evaluations of Generated Textcited by 52
  32. 2019EASSE: Easier Automatic Sentence Simplification Evaluationcited by 50
  33. 2018Importance of Search and Evaluation Strategies in Neural Dialogue Modelingcited by 44
  34. 2018Sentence-Level Fluency Evaluation: References Help, But Can Be Spared!cited by 43
  35. 2018Dual Conditional Cross-Entropy Filtering of Noisy Parallel Corporacited by 40
  36. 2021Conversations Are Not Flat: Modeling the Dynamic Information Flow across Dialogue Utterancescited by 40
  37. 2020GRUEN for Evaluating Linguistic Quality of Generated Textcited by 37
  38. 2021UMIC: An Unreferenced Metric for Image Captioning via Contrastive Learningcited by 37

50 most cited of 74 All 74 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 6; 2026: 3.

9 papers.

  1. 2025No Free Labels: Limitations of LLM-as-a-Judge Without Human Groundingcited by 54
  2. 2025Feeding Two Birds or Favoring One? Adequacy-Fluency Tradeoffs in Evaluation and Meta-Evaluation of Machine Translationcited by 4
  3. 2025Overview of the ClinIQLink 2025 Shared Task on Medical Question-Answeringcited by 3
  4. 2026Judging LLM-as-a-Judge: Concerning Rubric Artifacts in LLM-based Automated Text Generation Evaluationcited by 3
  5. 2025SpeechDialogueFactory: Generating High-Quality Speech Dialogue Data to Accelerate Your Speech-LLM Developmentcited by 1
  6. 2025A Conceptual Framework for AI Capability Evaluationscited by 0
  7. 2025Transparent Reference-free Automated Evaluation of Open-Ended User Survey Responsescited by 0
  8. 2026Fidelity-Diversity Metrics for Textcited by 0
  9. 2026Assessing Readability with LLMs: The Role of Reasoning and Few-Shot Promptingcited by 0