mlsanityData to 7 Oct 2026

BLEURT: Learning Robust Metrics for Text Generation

arXiv 2004.04696 ↗First posted 9 Apr 2020cs.CLcited by 1,196

Not on the map. Why?

Cited by 55 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Text generation has made significant advances in the last few years. Yet, evaluation metrics have lagged behind, as the most popular choices (e.g., BLEU and ROUGE) may correlate poorly with human judgments. We propose BLEURT, a learned evaluation metric based on BERT that can model human judgments with a few thousand possibly biased training examples. A key aspect of our approach is a novel pre-training scheme that uses millions of synthetic examples to help the model generalize. BLEURT provides state-of-the-art results on the last three years of the WMT Metrics shared task and the WebNLG Competition dataset. In contrast to a vanilla BERT-based approach, it yields superior results even when the training data is scarce and out-of-distribution.

Key facts. Cited by 1,196 papers in the map (data through 7 Oct 2026). First posted on arXiv 9 Apr 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2020: 1; 2021: 118; 2022: 185; 2023: 264; 2024: 298; 2025: 203; 2026: 127.

1,196 papers.

Sort:

By area:

  1. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  2. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  3. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  4. 2022Survey of Hallucination in Natural Language Generationcited by 2,056
  5. 2023PaLM 2 Technical Reportcited by 1,154
  6. 2023ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debatecited by 724
  7. 2021BARTScore: Evaluating Generated Text as Text Generationcited by 719
  8. 2021Cross-Task Generalization via Natural Language Crowdsourcing Instructionscited by 696
  9. 2023Prometheus: Inducing Fine-grained Evaluation Capability in Language Modelscited by 558
  10. 2023LLM-Blender: Ensembling Large Language Models with Pairwise Ranking and Generative Fusioncited by 550
  11. 2023Reinforced Self-Training (ReST) for Language Modelingcited by 486
  12. 2024From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judgecited by 480
38 more
  1. 2021MAUVE: Measuring the Gap Between Neural Text and Human Text using Divergence Frontierscited by 466
  2. 2023Benchmarking Large Language Models for News Summarizationcited by 444
  3. 2023Is ChatGPT a Good NLG Evaluator? A Preliminary Studycited by 420
  4. 2023Large Language Models Are State-of-the-Art Evaluators of Translation Qualitycited by 416
  5. 2024Contrastive Preference Optimization: Pushing the Boundaries of LLM Performance in Machine Translationcited by 390
  6. 2023GPTScore: Evaluate as You Desirecited by 357
  7. 2023Check Your Facts and Try Again: Improving Large Language Models with External Knowledge and Automated Feedbackcited by 351
  8. 2023JudgeLM: Fine-tuned Large Language Models are Scalable Judgescited by 351
  9. 2022One Embedder, Any Task: Instruction-Finetuned Text Embeddingscited by 329
  10. 2024Replacing Judges with Juries: Evaluating LLM Generations with a Panel of Diverse Modelscited by 301
  11. 2022Confident Adaptive Language Modelingcited by 297
  12. 2023Large Language Model Unlearningcited by 288
  13. 2023AlignScore: Evaluating Factual Consistency with a Unified Alignment Functioncited by 284
  14. 2024Is DPO Superior to PPO for LLM Alignment? A Comprehensive Studycited by 262
  15. 2023xCOMET: Transparent Machine Translation Evaluation through Fine-grained Error Detectioncited by 260
  16. 2024Self-Preference Bias in LLM-as-a-Judgecited by 248
  17. 2023Can Large Language Models Transform Computational Social Science?cited by 245
  18. 2022Is Reinforcement Learning (Not) for Natural Language Processing: Benchmarks, Baselines, and Building Blocks for Natural Language Policy Optimizationcited by 244
  19. 2021The GEM Benchmark: Natural Language Generation, its Evaluation and Metricscited by 239
  20. 2023Automatically Correcting Large Language Models: Surveying the landscape of diverse self-correction strategiescited by 236
  21. 2023L-Eval: Instituting Standardized Evaluation for Long Context Language Modelscited by 232
  22. 2024LLMs Know More Than They Show: On the Intrinsic Representation of LLM Hallucinationscited by 230
  23. 2021Get Your Vitamin C! Robust Fact Verification with Contrastive Evidencecited by 222
  24. 2024VLM2Vec: Training Vision-Language Models for Massive Multimodal Embedding Taskscited by 222
  25. 2024Tower: An Open Multilingual Large Language Model for Translation-Related Taskscited by 205
  26. 2023Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificitycited by 204
  27. 2024When Scaling Meets LLM Finetuning: The Effect of Data, Model and Finetuning Methodcited by 201
  28. 2022Co-Writing Screenplays and Theatre Scripts with Language Models: An Evaluation by Industry Professionalscited by 191
  29. 2023Does Writing with Language Models Reduce Content Diversity?cited by 190
  30. 2023ReMax: A Simple, Effective, and Efficient Reinforcement Learning Method for Aligning Large Language Modelscited by 184
  31. 2022ROSCOE: A Suite of Metrics for Scoring Step-by-Step Reasoningcited by 175
  32. 2023Time Travel in LLMs: Tracing Data Contamination in Large Language Modelscited by 175
  33. 2021FeTaQA: Free-form Table Question Answeringcited by 174
  34. 2023Benchmarking Foundation Models with Language-Model-as-an-Examinercited by 172
  35. 2021Hurdles to Progress in Long-form Question Answeringcited by 166
  36. 2022Chart-to-Text: A Large-Scale Benchmark for Chart Summarizationcited by 166
  37. 2023Large Language Model Routing with Benchmark Datasetscited by 166
  38. 2021To Ship or Not to Ship: An Extensive Evaluation of Automatic Metrics for Machine Translationcited by 161

50 most cited of 1,196 All 1,196 (JSON)