mlsanityData to 14 Oct 2026

All That's 'Human' Is Not Gold: Evaluating Human Evaluation of Generated Text

arXiv 2107.00061 ↗First posted 30 Jun 2021cs.CLcited by 287

Not on the map. Why?

Abstract (arXiv)

Human evaluations are typically considered the gold standard in natural language generation, but as models' fluency improves, how well can evaluators detect and judge machine-generated text? We run a study assessing non-experts' ability to distinguish between human- and machine-authored text (GPT2 and GPT3) in three domains (stories, news articles, and recipes). We find that, without training, evaluators distinguished between GPT3- and human-authored text at random chance level. We explore three approaches for quickly training evaluators to better identify GPT3-authored text (detailed instructions, annotated examples, and paired examples) and find that while evaluators' accuracy improved up to 55%, it did not significantly improve across the three domains. Given the inconsistent results across text domains and the often contradictory reasons evaluators gave for their judgments, we examine the role untrained human evaluations play in NLG evaluation and provide recommendations to NLG researchers for improving human evaluations of text generated from state-of-the-art models.

Key facts. Cited by 287 papers in the map (data through 14 Oct 2026). Builds on 18 of its 45 references that are in the map. First posted on arXiv 30 Jun 2021.

Builds on

18 of 45 references.

References · 18

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,100
  3. 2018Hierarchical Neural Story Generationcited by 1,193
  4. 2019PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarizationcited by 1,102
  5. 2019Defending Against Neural Fake Newscited by 559
  6. 2019GLTR: Statistical Detection and Visualization of Generated Textcited by 496
  7. 2016How NOT To Evaluate Your Dialogue System: An Empirical Study of Unsupervised Evaluation Metrics for Dialogue Response Generationcited by 356
  8. 2019Automatic Detection of Generated Text is Easiest when Humans are Fooledcited by 261
  9. 2021The GEM Benchmark: Natural Language Generation, its Evaluation and Metricscited by 239
  10. 2017Why We Need New Evaluation Metrics for NLGcited by 233
  11. 2020With Little Power Comes Great Responsibilitycited by 142
  12. 2019Unifying Human and Statistical Evaluation for Natural Language Generationcited by 127
6 more
  1. 2018RankME: Reliable Human Ratings for Natural Language Generationcited by 69
  2. 2020STORIUM: A Dataset and Evaluation Platform for Machine-in-the-Loop Story Generationcited by 56
  3. 2020RoFT: A Tool for Evaluating Human Detection of Machine-Generated Textcited by 36
  4. 2019Judge the Judges: A Large-Scale Evaluation Study of Neural Language Models for Online Review Generationcited by 17
  5. 2019Towards Best Experiment Design for Evaluating Dialogue System Outputcited by 15
  6. 2016Mechanical Novel: Crowdsourcing Complex Work through Reflection and Revisioncited by 13

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 10; 2022: 44; 2023: 71; 2024: 58; 2025: 70; 2026: 34.

287 papers.

Sort:

By area:

  1. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  2. 2023Can Large Language Models Be an Alternative to Human Evaluations?cited by 734
  3. 2022ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detectioncited by 422
  4. 2022News Summarization and Evaluation in the Era of GPT-3cited by 327
  5. 2024Humans or LLMs as the Judge? A Study on Judgement Biasescited by 323
  6. 2024PKU-SafeRLHF: Towards Multi-Level Safety Alignment for LLMs with Human Preferencecited by 261
  7. 2023Automatically Correcting Large Language Models: Surveying the landscape of diverse self-correction strategiescited by 238
  8. 2022Quark: Controllable Text Generation with Reinforced Unlearningcited by 237
  9. 2022Neural Theory-of-Mind? On the Limits of Social Intelligence in Large LMscited by 220
  10. 2022WANLI: Worker and AI Collaboration for Natural Language Inference Dataset Creationcited by 202
  11. 2023GPT detectors are biased against non-native English writerscited by 178
  12. 2023Can LLM-Generated Misinformation Be Detected?cited by 177
38 more
  1. 2023Art or Artifice? Large Language Models and the False Promise of Creativitycited by 172
  2. 2024Monitoring AI-Modified Content at Scale: A Case Study on the Impact of ChatGPT on AI Conference Peer Reviewscited by 164
  3. 2024RAID: A Shared Benchmark for Robust Evaluation of Machine-Generated Text Detectorscited by 163
  4. 2023UniChart: A Universal Vision-language Pretrained Model for Chart Comprehension and Reasoningcited by 160
  5. 2022Repairing the Cracked Foundation: A Survey of Obstacles in Evaluation Practices for Generated Textcited by 155
  6. 2022Revisiting the Gold Standard: Grounding Summarization Evaluation with Robust Human Evaluationcited by 142
  7. 2023The Science of Detecting LLM-Generated Textscited by 128
  8. 2023On the Risk of Misinformation Pollution with Large Language Modelscited by 128
  9. 2023AI Transparency in the Age of LLMs: A Human-Centered Research Roadmapcited by 123
  10. 2023M4: Multi-generator, Multi-domain, and Multi-lingual Black-Box Machine-Generated Text Detectioncited by 121
  11. 2024Relying on the Unreliable: The Impact of Language Models' Reluctance to Express Uncertaintycited by 106
  12. 2022Human heuristics for AI-generated language are flawedcited by 105
  13. 2023A Survey on LLM-Generated Text Detection: Necessity, Methods, and Future Directionscited by 105
  14. 2023Synthetic Data Generation with Large Language Models for Text Classification: Potential and Limitationscited by 100
  15. 2023A Confederacy of Models: a Comprehensive Evaluation of LLMs on Creative Writingcited by 99
  16. 2021The Perils of Using Mechanical Turk to Evaluate Open-Ended Text Generationcited by 97
  17. 2023LongEval: Guidelines for Human Evaluation of Faithfulness in Long-form Summarizationcited by 96
  18. 2023Language Model Behavior: A Comprehensive Surveycited by 92
  19. 2022On Decoding Strategies for Neural Text Generatorscited by 90
  20. 2021Is GPT-3 Text Indistinguishable from Human Text? Scarecrow: A Framework for Scrutinizing Machine Textcited by 89
  21. 2023Watermarks in the Sand: Impossibility of Strong Watermarking for Generative Modelscited by 87
  22. 2023Human Feedback is not Gold Standardcited by 83
  23. 2023A Survey of GPT-3 Family Large Language Models Including ChatGPT and GPT-4cited by 81
  24. 2022Machine Generated Text: A Comprehensive Survey of Threat Models and Detection Methodscited by 79
  25. 2023AI model GPT-3 (dis)informs us better than humanscited by 78
  26. 2025Learning to Plan & Reason for Evaluation with Thinking-LLM-as-a-Judgecited by 78
  27. 2024A Design Space for Intelligent and Interactive Writing Assistantscited by 76
  28. 2023EvalLM: Interactive Evaluation of Large Language Model Prompts on User-Defined Criteriacited by 74
  29. 2022Do Language Models Plagiarize?cited by 73
  30. 2022RankGen: Improving Text Generation with Large Ranking Modelscited by 72
  31. 2023Style Over Substance: Evaluation Biases for Large Language Modelscited by 70
  32. 2022Mutual Information Alleviates Hallucinations in Abstractive Summarizationcited by 68
  33. 2022Real or Fake Text?: Investigating Human Ability to Detect Boundaries Between Human-Written and Machine-Generated Textcited by 63
  34. 2023The Next Chapter: A Study of Large Language Models in Storytellingcited by 58
  35. 2021A Systematic Investigation of Commonsense Knowledge in Large Language Modelscited by 56
  36. 2023Rethinking Model Evaluation as Narrowing the Socio-Technical Gapcited by 56
  37. 2023Bridging the Gap: A Survey on Integrating (Human) Feedback for Natural Language Generationcited by 55
  38. 2023Why So Gullible? Enhancing the Robustness of Retrieval-Augmented Models against Counterfactual Noisecited by 54

50 most cited of 287 All 287 (JSON)