mlsanityData to 7 Oct 2026

On the Evaluation Metrics for Paraphrase Generation

arXiv 2202.08479 ↗First posted 17 Feb 2022cs.CL cs.AIcited by 32

Not on the map. Why?

Abstract (arXiv)

In this paper we revisit automatic metrics for paraphrase evaluation and obtain two findings that disobey conventional wisdom: (1) Reference-free metrics achieve better performance than their reference-based counterparts. (2) Most commonly used metrics do not align well with human annotation. Underlying reasons behind the above findings are explored through additional experiments and in-depth analyses. Based on the experiments and analyses, we propose ParaScore, a new evaluation metric for paraphrase generation. It possesses the merits of reference-based and reference-free metrics and explicitly models lexical divergence. Experimental results demonstrate that ParaScore significantly outperforms existing metrics.

Key facts. Cited by 32 papers in the map (data through 7 Oct 2026). Builds on 28 of its 63 references that are in the map. First posted on arXiv 17 Feb 2022.

Builds on

28 of 63 references.

References · 28

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  3. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  4. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,168
  5. 2020BLEURT: Learning Robust Metrics for Text Generationcited by 1,196
  6. 2021BARTScore: Evaluating Generated Text as Text Generationcited by 719
  7. 2019MoverScore: Text Generation Evaluating with Contextualized Embeddings and Earth Mover Distancecited by 415
  8. 2021A Survey of Data Augmentation Approaches for NLPcited by 408
  9. 2020BLEU might be Guilty but References are not Innocentcited by 109
  10. 2021Neural Machine Translation with Monolingual Translation Memorycited by 61
  11. 2020Neural Syntactic Preordering for Controlled Paraphrase Generationcited by 56
  12. 2020Syntax-guided Controlled Generation of Paraphrasescited by 47
16 more
  1. 2017Learning to Paraphrase for Question Answeringcited by 46
  2. 2021Generating Syntactically Controlled Paraphrases without Using Annotated Parallel Pairscited by 45
  3. 2019Unsupervised Paraphrasing by Simulated Annealingcited by 37
  4. 2018ComQA: A Community-sourced Dataset for Complex Factoid Question Answering with Paraphrase Clusterscited by 29
  5. 2020Unsupervised Paraphrasing via Deep Reinforcement Learningcited by 25
  6. 2019Don't paraphrase, detect! Rapid and Effective Data Collection for Semantic Parsingcited by 24
  7. 2018Neural Network Models for Paraphrase Identification, Semantic Textual Similarity, Natural Language Inference, and Question Answeringcited by 23
  8. 2021A Training-free and Reference-free Summarization Evaluation Metric via Centrality-weighted Relevance and Self-referenced Redundancycited by 21
  9. 2020Unsupervised Dual Paraphrasing for Two-stage Semantic Parsingcited by 20
  10. 2018Automatic Metric Validation for Grammatical Error Correctioncited by 17
  11. 2021Assessing Dialogue Systems with Distribution Distancescited by 16
  12. 2020Unsupervised Paraphrase Generation using Pre-trained Language Modelscited by 15
  13. 2020Unsupervised Paraphrasing with Pretrained Language Modelscited by 15
  14. 2021From Paraphrasing to Semantic Parsing: Unsupervised Semantic Parsing via Synchronous Semantic Decodingcited by 14
  15. 2018NEXUS Network: Connecting the Preceding and the Following in Dialogue Generationcited by 9
  16. 2021REAMsharp: An Enhancement Approach to Reference-based Evaluation Metrics for Open-domain Dialog Generationcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 8; 2024: 12; 2025: 7; 2026: 3.

32 papers.

Sort:

By area:

  1. 2023Exploring the Use of Large Language Models for Reference-Free Text Quality Evaluation: An Empirical Studycited by 85
  2. 2022On the Blind Spots of Model-Based Evaluation Metrics for Text Generationcited by 48
  3. 2023Fusing Models with Complementary Expertisecited by 45
  4. 2023The Trickle-down Impact of Reward (In-)consistency on RLHFcited by 29
  5. 2024Themis: A Reference-free NLG Evaluation Language Model with Flexibility and Interpretabilitycited by 19
  6. 2024Text Generation: A Systematic Literature Review of Tasks, Evaluation, and Challengescited by 18
  7. 2023Building Trustworthy NeuroSymbolic AI Systems: Consistency, Reliability, Explainability, and Safetycited by 14
  8. 2024DnA-Eval: Enhancing Large Language Model Evaluation through Decomposition and Aggregationcited by 12
  9. 2024LAMPAT: Low-Rank Adaption for Multilingual Paraphrasing Using Adversarial Trainingcited by 8
  10. 2025AdParaphrase: Paraphrase Dataset for Analyzing Linguistic Features toward Generating Attractive Ad Textscited by 5
  11. 2024Spivavtor: An Instruction Tuned Ukrainian Text Editing Modelcited by 4
  12. 2024Do Audio-Language Models Understand Linguistic Variations?cited by 4
20 more
  1. 2024Measuring Moral Inconsistencies in Large Language Modelscited by 3
  2. 2024SaGE: Evaluating Moral Consistency in Large Language Modelscited by 3
  3. 2024PlagBench: Exploring the Duality of Large Language Models in Plagiarism Generation and Detectioncited by 3
  4. 2025The Ultimate Cookbook for Invisible Poison: Crafting Subtle Clean-Label Text Backdoors with Style Attributescited by 2
  5. 2025Evaluating Text Style Transfer: A Nine-Language Benchmark for Text Detoxificationcited by 2
  6. 2022Dense Paraphrasing for Textual Enrichmentcited by 1
  7. 2023Sen2Pro: A Probabilistic Perspective to Sentence Embedding from Pre-trained Language Modelcited by 1
  8. 2023A Quality-based Syntactic Template Retriever for Syntactically-controlled Paraphrase Generationcited by 1
  9. 2024Improved Paraphrase Generation via Controllable Latent Diffusioncited by 1
  10. 2024Linguistically-Controlled Paraphrase Generationcited by 1
  11. 2025Context Selection and Rewriting for Video-based Educational Question Generationcited by 1
  12. 2025Retrieval Augmented Generation-based Large Language Models for Bridging Transportation Cybersecurity Legal Knowledge Gapscited by 1
  13. 2025Music Recommendation with Large Language Models: Challenges, Opportunities, and Evaluationcited by 1
  14. 2023IdEALS: Idiomatic Expressions for Advancement of Language Skillscited by 0
  15. 2023A Benchmark for Text Expansion: Datasets, Metrics, and Baselinescited by 0
  16. 2024AMPS: ASR with Multimodal Paraphrase Supervisioncited by 0
  17. 2025Enhancing Paraphrase Type Generation: The Impact of DPO and RLHF Evaluated with Human-Ranked Datacited by 0
  18. 2026Helping the Helper: LLM-Assisted Problem Articulation for Older Adults Seeking Technology Supportcited by 0
  19. 2026Beyond a Single Reference: Training and Evaluation with Paraphrases in Sign Language Translationcited by 0
  20. 2026Short-form Text Rewriting with Phi Silicacited by 0