mlsanityData to 7 Oct 2026

LENS: A Learnable Evaluation Metric for Text Simplification

arXiv 2212.09739 ↗First posted 19 Dec 2022cs.CLcited by 54

Not on the map. Why?

Abstract (arXiv)

Training learnable metrics using modern language models has recently emerged as a promising method for the automatic evaluation of machine translation. However, existing human evaluation datasets for text simplification have limited annotations that are based on unitary or outdated models, making them unsuitable for this approach. To address these issues, we introduce the SimpEval corpus that contains: SimpEval_past, comprising 12K human ratings on 2.4K simplifications of 24 past systems, and SimpEval_2022, a challenging simplification benchmark consisting of over 1K human ratings of 360 simplifications including GPT-3.5 generated text. Training on SimpEval, we present LENS, a Learnable Evaluation Metric for Text Simplification. Extensive empirical results show that LENS correlates much better with human judgment than existing metrics, paving the way for future progress in the evaluation of text simplification. We also introduce Rank and Rate, a human evaluation framework that rates simplifications from several models in a list-wise manner using an interactive interface, which ensures both consistency and accuracy in the evaluation process and is used to create the SimpEval datasets.

Key facts. Cited by 54 papers in the map (data through 7 Oct 2026). Builds on 57 of its 106 references that are in the map. First posted on arXiv 19 Dec 2022.

Builds on

57 of 106 references.

References · 57

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  5. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  6. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  7. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  8. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  9. 2020BLEURT: Learning Robust Metrics for Text Generationcited by 1,196
  10. 2020COMET: A Neural Framework for MT Evaluationcited by 1,024
  11. 2015Sequence Level Training with Recurrent Neural Networkscited by 568
  12. 2019MoverScore: Text Generation Evaluating with Contextualized Embeddings and Earth Mover Distancecited by 415
38 more
  1. 2022News Summarization and Evaluation in the Era of GPT-3cited by 324
  2. 2016Deep Reinforcement Learning for Dialogue Generationcited by 286
  3. 2017Toward Controlled Generation of Textcited by 271
  4. 2018Dear Sir or Madam, May I introduce the GYAFC Dataset: Corpus, Benchmarks and Metrics for Formality Style Transfercited by 209
  5. 2019Leveraging Pre-trained Checkpoints for Sequence Generation Taskscited by 182
  6. 2020Reformulating Unsupervised Style Transfer as Paraphrase Generationcited by 181
  7. 2021To Ship or Not to Ship: An Extensive Evaluation of Automatic Metrics for Machine Translationcited by 161
  8. 2017Unsupervised Neural Machine Translationcited by 148
  9. 2019What makes a good conversation? How controllable attributes affect human judgmentscited by 140
  10. 2019Unifying Human and Statistical Evaluation for Natural Language Generationcited by 127
  11. 2018Style Transfer Through Back-Translationcited by 124
  12. 2015Minimum Risk Training for Neural Machine Translationcited by 114
  13. 2019On NMT Search Errors and Model Errors: Cat Got Your Tongue?cited by 114
  14. 2020If beam search is the answer, what was the question?cited by 114
  15. 2017Sentence Simplification with Deep Reinforcement Learningcited by 107
  16. 2020Neural CRF Model for Sentence Alignment in Text Simplificationcited by 106
  17. 2019Automatically Neutralizing Subjective Bias in Textcited by 105
  18. 2021High Quality Rather than High Model Probability: Minimum Bayes Risk Decoding with Neural Metricscited by 104
  19. 2020ASSET: A Dataset for Tuning and Evaluation of Sentence Simplification Models with Multiple Rewriting Transformationscited by 101
  20. 2021Decontextualization: Making Sentences Stand-Alonecited by 101
  21. 2021Is GPT-3 Text Indistinguishable from Human Text? Scarecrow: A Framework for Scrutinizing Machine Textcited by 89
  22. 2019Beyond BLEU: Training Neural Machine Translation with Semantic Similaritycited by 84
  23. 2019Controllable Sentence Simplificationcited by 78
  24. 2022RankGen: Improving Text Generation with Large Ranking Modelscited by 72
  25. 2021A Statistical Analysis of Summarization Evaluation Metrics using Resampling Methodscited by 71
  26. 2018RankME: Reliable Human Ratings for Natural Language Generationcited by 67
  27. 2019EditNTS: An Neural Programmer-Interpreter Model for Sentence Simplification through Explicit Editingcited by 63
  28. 2019Evaluating Style Transfer for Textcited by 59
  29. 2020PowerTransformer: Unsupervised Controllable Revision for Biased Language Correctioncited by 59
  30. 2022Understanding Iterative Revision from Human-Written Textcited by 56
  31. 2020Controllable Text Simplification with Explicit Paraphrasingcited by 53
  32. 2022Quality-Aware Decoding for Neural Machine Translationcited by 49
  33. 2020Energy-Based Reranking: Improving Neural Machine Translation Using Energy-Based Modelscited by 44
  34. 2022Identifying Weaknesses in Machine Translation Metrics Through Minimum Bayes Risk Decoding: A Case Study for COMETcited by 41
  35. 2019Reinforcement Learning Based Text Style Transfer without Parallel Training Corpuscited by 38
  36. 2022Re-Examining System-Level Correlations of Automatic Summarization Evaluation Metricscited by 37
  37. 2020Iterative Edit-Based Unsupervised Sentence Simplificationcited by 33
  38. 2018Semantic Structural Evaluation for Text Simplificationcited by 30

50 most cited of 57 All 57 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 12; 2024: 20; 2025: 13; 2026: 9.

54 papers.

Sort:

By area:

  1. 2024Foundational Autoraters: Taming Large Language Models for Better Automatic Evaluationcited by 72
  2. 2024Can AI writing be salvaged? Mitigating Idiosyncrasies and Improving Human-AI Alignment in the Writing Process through Editscited by 49
  3. 2023Reducing Privacy Risks in Online Self-Disclosures with Language Modelscited by 47
  4. 2024Polos: Multimodal Metric Learning from Human Feedback for Image Captioningcited by 46
  5. 2023Beyond the Chat: Executable and Verifiable Text-Editing with LLMscited by 35
  6. 2023BLESS: Benchmarking Large Language Models on Sentence Simplificationcited by 34
  7. 2024RAG-RLRC-LaySum at BioLaySumm: Integrating Retrieval-Augmented Generation and Readability Control for Layman Summarization of Biomedical Textscited by 23
  8. 2023ReadMe++: Benchmarking Multilingual Language Models for Multi-Domain Readability Assessmentcited by 19
  9. 2024Themis: A Reference-free NLG Evaluation Language Model with Flexibility and Interpretabilitycited by 19
  10. 2024Automatic Metrics in Natural Language Generation: A Survey of Current Evaluation Practicescited by 19
  11. 2024Overview of the BioLaySumm 2024 Shared Task on the Lay Summarization of Biomedical Research Articlescited by 18
  12. 2023Flesch or Fumble? Evaluating Readability Standard Alignment of Instruction-Tuned Language Modelscited by 17
38 more
  1. 2023Do Text Simplification Systems Preserve Meaning? A Human Evaluation via Reading Comprehensioncited by 16
  2. 2023APPLS: Evaluating Evaluation Metrics for Plain Language Summarizationcited by 15
  3. 2023Dancing Between Success and Failure: Edit-level Simplification Evaluation using SALSAcited by 15
  4. 2023Simplicity Level Estimate (SLE): A Learned Reference-Less Metric for Sentence Simplificationcited by 14
  5. 2024Improving Minimum Bayes Risk Decoding with Multi-Promptcited by 13
  6. 2024DENEB: A Hallucination-Robust Automatic Evaluation Metric for Image Captioningcited by 11
  7. 2023Thresh: A Unified, Customizable and Deployable Platform for Fine-Grained Text Evaluationcited by 6
  8. 2023Robustness Tests for Automatic Machine Translation Metrics with Adversarial Attackscited by 5
  9. 2025Summarizing Speech: A Comprehensive Surveycited by 5
  10. 2023Automatic and Human-AI Interactive Text Generationcited by 4
  11. 2024German Text Simplification: Finetuning Large Language Models with Semi-Synthetic Datacited by 4
  12. 2024An In-depth Evaluation of Large Language Models in Sentence Simplification with Error-based Human Assessmentcited by 4
  13. 2024Semantic Graphs for Syntactic Simplification: A Revisit from the Age of LLMcited by 4
  14. 2025OpeNLGauge: An Explainable Metric for NLG Evaluation with Open-Weights LLMscited by 4
  15. 2025Evaluation Under Imperfect Benchmarks and Ratings: A Case Study in Text Simplificationcited by 4
  16. 2025LLM-based Text Simplification and its Effect on User Comprehension and Cognitive Loadcited by 4
  17. 2024Digital Comprehensibility Assessment of Simplified Texts among Persons with Intellectual Disabilitiescited by 3
  18. 2024REFeREE: A REference-FREE Model-Based Metric for Text Simplificationcited by 3
  19. 2024Label Confidence Weighted Learning for Target-level Sentence Simplificationcited by 3
  20. 2024SiTSE: Sinhala Text Simplification Dataset and Evaluationcited by 3
  21. 2024M-Ped: Multi-Prompt Ensemble Decoding for Large Language Modelscited by 3
  22. 2025Aligning Sentence Simplification with ESL Learner's Proficiency for Language Acquisitioncited by 3
  23. 2025Are LLM-generated plain language summaries truly understandable? A large-scale crowdsourced evaluationcited by 2
  24. 2024WisPerMed at BioLaySumm: Adapting Autoregressive Large Language Models for Lay Summarization of Scientific Articlescited by 1
  25. 2024MaLei at the PLABA Track of TREC 2024: RoBERTa for Term Replacement -- LLaMA3.1 and GPT-4o for Complete Abstract Adaptationcited by 1
  26. 2025Improving Text Style Transfer using Masked Diffusion Language Models with Inference-time Scalingcited by 1
  27. 2025PERCS: Persona-Guided Controllable Biomedical Summarization Datasetcited by 1
  28. 2025Policy-based Sentence Simplification: Replacing Parallel Corpora with LLM-as-a-Judgecited by 1
  29. 2026Simplify-This: A Comparative Analysis of Prompt-Based and Fine-Tuned LLMscited by 1
  30. 2026A Human-in/on-the-Loop Framework for Accessible Text Generationcited by 1
  31. 2026MedLayBench-V: A Large-Scale Benchmark for Expert-Lay Semantic Alignment in Medical Vision Language Modelscited by 1
  32. 2024EASSE-DE: Easier Automatic Sentence Simplification Evaluation for Germancited by 0
  33. 2025Evaluating the Effectiveness of Direct Preference Optimization for Personalizing German Automatic Text Simplifications for Persons with Intellectual Disabilitiescited by 0
  34. 2025JUDGEBERT: Assessing Legal Meaning Preservation Between Sentencescited by 0
  35. 2025DETECT: Determining Ease and Textual Clarity of German Text Simplificationscited by 0
  36. 2025Text Simplification with Sentence Embeddingscited by 0
  37. 2026Profiling German Text Simplification with Interpretable Model-Fingerprintscited by 0
  38. 2026Taming CATS: Controllable Automatic Text Simplification through Instruction Fine-Tuning with Control Tokenscited by 0

50 most cited of 54 All 54 (JSON)