mlsanityData to 7 Oct 2026

Sentence-Level Fluency Evaluation: References Help, But Can Be Spared!

arXiv 1809.08731 ↗First posted 24 Sep 2018cs.CLcited by 43

Not on the map. Why?

Abstract (arXiv)

Motivated by recent findings on the probabilistic modeling of acceptability judgments, we propose syntactic log-odds ratio (SLOR), a normalized language model score, as a metric for referenceless fluency evaluation of natural language generation output at the sentence level. We further introduce WPSLOR, a novel WordPiece-based version, which harnesses a more compact language model. Even though word-overlap metrics like ROUGE are computed with the help of hand-written references, our referenceless methods obtain a significantly higher correlation with human fluency scores on a benchmark dataset of compressed sentences. Finally, we present ROUGE-LM, a reference-based metric which is a natural extension of WPSLOR to the case of available references. We show that ROUGE-LM yields a significantly higher correlation with human judgments than all baseline metrics, including WPSLOR on its own.

Key facts. Cited by 43 papers in the map (data through 7 Oct 2026). First posted on arXiv 24 Sep 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 6; 2022: 8; 2023: 11; 2024: 11; 2025: 3; 2026: 4.

43 papers.

Sort:

By area:

  1. 2022Understanding Iterative Revision from Human-Written Textcited by 56
  2. 2021Keep it Simple: Unsupervised Simplification of Multi-Paragraph Textcited by 45
  3. 2021LM-Critic: Language Models for Unsupervised Grammatical Error Correctioncited by 33
  4. 2022EditEval: An Instruction-Based Benchmark for Text Improvementscited by 30
  5. 2022Phrase-level Textual Adversarial Attack with Label Preservationcited by 23
  6. 2023Towards Imperceptible Document Manipulations against Neural Ranking Modelscited by 18
  7. 2022TASA: Deceiving Question Answering Models by Twin Answer Sentences Attackcited by 15
  8. 2023Computational Language Acquisition with Theory of Mindcited by 15
  9. 2023AGIR: Automating Cyber Threat Intelligence Reporting with Natural Language Generationcited by 15
  10. 2021NAREOR: The Narrative Reordering Problemcited by 14
  11. 2024How to Make the Most of LLMs' Grammatical Knowledge for Acceptability Judgmentscited by 14
  12. 2023Do LLMs Work on Charts? Designing Few-Shot Prompts for Chart Question Answering and Summarizationcited by 10
31 more
  1. 2024What Goes Into a LM Acceptability Judgment? Rethinking the Impact of Frequency and Lengthcited by 10
  2. 2025Reference-free Evaluation Metrics for Text Generation: A Surveycited by 9
  3. 2024A Survey of Data Synthesis Approachescited by 8
  4. 2021Meta-evaluation of Conversational Search Evaluation Metricscited by 6
  5. 2024ToBlend: Token-Level Blending With an Ensemble of LLMs to Attack AI-Generated Text Detectioncited by 6
  6. 2024Is Reference Necessary in the Evaluation of NLG Systems? When and Where?cited by 6
  7. 2024RepEval: Effective Text Evaluation with LLM Representationcited by 6
  8. 2024Large Language Models Are Active Critics in NLG Evaluationcited by 6
  9. 2024Attention Speaks Volumes: Localizing and Mitigating Bias in Language Modelscited by 6
  10. 2022Improving Zero-Shot Event Extraction via Sentence Simplificationcited by 5
  11. 2022Rethinking Textual Adversarial Defense for Pre-trained Language Modelscited by 5
  12. 2022Controllable Citation Sentence Generation with Language Modelscited by 4
  13. 2023Large Language Models, scientific knowledge and factuality: A framework to streamline human expert evaluationcited by 4
  14. 2023Controllable Generation of Dialogue Acts for Dialogue Systems via Few-Shot Response Generation and Rankingcited by 4
  15. 2021Graphmax for Text Generationcited by 2
  16. 2021On Neurons Invariant to Sentence Structural Changes in Neural Machine Translationcited by 2
  17. 2023System Combination via Quality Estimation for Grammatical Error Correctioncited by 2
  18. 2023Unsupervised Approach to Evaluate Sentence-Level Fluency: Do We Really Need Reference?cited by 2
  19. 2023A Multilingual Virtual Guide for Self-Attachment Techniquecited by 1
  20. 2024SITransformer: Shared Information-Guided Transformer for Extreme Multimodal Summarizationcited by 1
  21. 2025Eye Tracking Based Cognitive Evaluation of Automatic Readability Assessment Methodscited by 1
  22. 2026Augmenting Text to Increase Translation Difficultycited by 1
  23. 2022An Empirical Study on Learning and Improving the Search Objective for Unsupervised Paraphrasingcited by 0
  24. 2023Efficient and Interpretable Compressive Text Summarisation with Unsupervised Dual-Agent Reinforcement Learningcited by 0
  25. 2023Can Knowledge Graphs Simplify Text?cited by 0
  26. 2024Reversible Jump Attack to Textual Classifiers with Modification Reductioncited by 0
  27. 2024Thesis proposal: Are We Losing Textual Diversity to Natural Language Processing?cited by 0
  28. 2025ContextBench: Modifying Contexts for Targeted Latent Activationcited by 0
  29. 2026Output Composability of QLoRA PEFT Modules for Plug-and-Play Attribute-Controlled Text Generationcited by 0
  30. 2026A Comparative Study of Controlled Text Generation Systems Using Level-Playing-Field Evaluation Principlescited by 0
  31. 2026Readers make targeted regressions to plausible errors in reanalysis of "noisy-channel garden-path" sentencescited by 0