mlsanityData to 7 Oct 2026

BERTScore: Evaluating Text Generation with BERT

arXiv 1904.09675 ↗First posted 21 Apr 2019cs.CLcited by 5,462

Not on the map. Why?

Cited by 296 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We propose BERTScore, an automatic evaluation metric for text generation. Analogously to common metrics, BERTScore computes a similarity score for each token in the candidate sentence with each token in the reference sentence. However, instead of exact matches, we compute token similarity using contextual embeddings. We evaluate using the outputs of 363 machine translation and image captioning systems. BERTScore correlates better with human judgments and provides stronger model selection performance than existing metrics. Finally, we use an adversarial paraphrase detection task to show that BERTScore is more robust to challenging examples when compared to existing metrics.

Key facts. Cited by 5,462 papers in the map (data through 7 Oct 2026). First posted on arXiv 21 Apr 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2020: 1; 2021: 328; 2022: 509; 2023: 848; 2024: 1343; 2025: 1387; 2026: 1046.

5,462 papers.

Sort:

By area:

  1. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  2. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,388
  3. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,205
  4. 2022Survey of Hallucination in Natural Language Generationcited by 2,056
  5. 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,259
  6. 2023A Survey on Evaluation of Large Language Modelscited by 1,227
  7. 2023Large Language Models are not Fair Evaluatorscited by 1,050
  8. 2023Training Diffusion Models with Reinforcement Learningcited by 939
  9. 2023A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivitycited by 888
  10. 2024Length-Controlled AlpacaEval: A Simple Way to Debias Automatic Evaluatorscited by 865
  11. 2024Evaluating Very Long-Term Conversational Memory of LLM Agentscited by 800
  12. 2023Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formattingcited by 778
38 more
  1. 2023ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debatecited by 724
  2. 2021BARTScore: Evaluating Generated Text as Text Generationcited by 719
  3. 2023Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Modelscited by 695
  4. 2023Prometheus: Inducing Fine-grained Evaluation Capability in Language Modelscited by 558
  5. 2023LLM-Blender: Ensembling Large Language Models with Pairwise Ranking and Generative Fusioncited by 550
  6. 2023MotionGPT: Human Motion as a Foreign Languagecited by 521
  7. 2024From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judgecited by 480
  8. 2021MAUVE: Measuring the Gap Between Neural Text and Human Text using Divergence Frontierscited by 466
  9. 2023Benchmarking Large Language Models for News Summarizationcited by 444
  10. 2022DiffuSeq: Sequence to Sequence Text Generation with Diffusion Modelscited by 439
  11. 2024Prometheus 2: An Open Source Language Model Specialized in Evaluating Other Language Modelscited by 431
  12. 2023LaMP: When Large Language Models Meet Personalizationcited by 425
  13. 2023Is ChatGPT a Good NLG Evaluator? A Preliminary Studycited by 420
  14. 2023Ragas: Automated Evaluation of Retrieval Augmented Generationcited by 408
  15. 2022Unnatural Instructions: Tuning Language Models with (Almost) No Human Laborcited by 395
  16. 2024Panda-70M: Captioning 70M Videos with Multiple Cross-Modality Teacherscited by 389
  17. 2023Med-Flamingo: a Multimodal Medical Few-shot Learnercited by 387
  18. 2024Hallucination of Multimodal Large Language Models: A Surveycited by 380
  19. 2022RLPrompt: Optimizing Discrete Text Prompts with Reinforcement Learningcited by 361
  20. 2022TM2T: Stochastic and Tokenized Modeling for the Reciprocal Generation of 3D Human Motions and Textscited by 357
  21. 2023GPTScore: Evaluate as You Desirecited by 357
  22. 2023Check Your Facts and Try Again: Improving Large Language Models with External Knowledge and Automated Feedbackcited by 351
  23. 2023JudgeLM: Fine-tuned Large Language Models are Scalable Judgescited by 351
  24. 2022One Embedder, Any Task: Instruction-Finetuned Text Embeddingscited by 329
  25. 2023HaluEval: A Large-Scale Hallucination Evaluation Benchmark for Large Language Modelscited by 329
  26. 2022News Summarization and Evaluation in the Era of GPT-3cited by 324
  27. 2023Analyzing and Mitigating Object Hallucination in Large Vision-Language Modelscited by 314
  28. 2024Humans or LLMs as the Judge? A Study on Judgement Biasescited by 314
  29. 2023PandaLM: An Automatic Evaluation Benchmark for LLM Instruction Tuning Optimizationcited by 311
  30. 2023Evaluating Verifiability in Generative Search Enginescited by 297
  31. 2023ReConcile: Round-Table Conference Improves Reasoning via Consensus among Diverse LLMscited by 297
  32. 2023Large Language Model Unlearningcited by 288
  33. 2021Recursively Summarizing Books with Human Feedbackcited by 287
  34. 2021Understanding Factuality in Abstractive Summarization with FRANK: A Benchmark for Factuality Metricscited by 285
  35. 2023AlignScore: Evaluating Factual Consistency with a Unified Alignment Functioncited by 284
  36. 2023LLMLingua: Compressing Prompts for Accelerated Inference of Large Language Modelscited by 279
  37. 2022Towards a Unified Multi-Dimensional Evaluator for Text Generationcited by 264
  38. 2024Is DPO Superior to PPO for LLM Alignment? A Comprehensive Studycited by 262

50 most cited of 5,462 All 5,462 (JSON)