# Reference-less Quality Estimation of Text Simplification Systems

arXiv: https://arxiv.org/abs/1901.10746 · HTML: https://mlsanity.com/paper/1901.10746/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Jan 2019.

## Built on by (most cited first, 10 of 10)

- [On the Limitations of Reference-Free Evaluations of Generated Text](https://mlsanity.com/paper/2210.12563/) (2022)
- [Med-EASi: Finely Annotated Dataset and Models for Controllable Simplification of Medical Texts](https://mlsanity.com/paper/2302.09155/) (2023)
- [Simplicity Level Estimate (SLE): A Learned Reference-Less Metric for Sentence Simplification](https://mlsanity.com/paper/2310.08170/) (2023)
- [MaskEval: Weighted MLM-Based Evaluation for Text Summarization and Simplification](https://mlsanity.com/paper/2205.12394/) (2022)
- [REFeREE: A REference-FREE Model-Based Metric for Text Simplification](https://mlsanity.com/paper/2403.17640/) (2024)
- [Unsupervised Sentence Simplification via Dependency Parsing](https://mlsanity.com/paper/2206.12261/) (2022)
- [Readability-guided Idiom-aware Sentence Simplification (RISS) for Chinese](https://mlsanity.com/paper/2406.02974/) (2024)
- [LC-Score: Reference-less estimation of Text Comprehension Difficulty](https://mlsanity.com/paper/2310.02754/) (2023)
- [CEScore: Simple and Efficient Confidence Estimation Model for Evaluating Split and Rephrase](https://mlsanity.com/paper/2312.01356/) (2023)
- [EASSE-DE: Easier Automatic Sentence Simplification Evaluation for German](https://mlsanity.com/paper/2404.03563/) (2024)

## Abstract (arXiv)

The evaluation of text simplification (TS) systems remains an open challenge. As the task has common points with machine translation (MT), TS is often evaluated using MT metrics such as BLEU. However, such metrics require high quality reference data, which is rarely available for TS. TS has the advantage over MT of being a monolingual task, which allows for direct comparisons to be made between the simplified text and its original version. In this paper, we compare multiple approaches to reference-less quality estimation of sentence-level text simplification systems, based on the dataset used for the QATS 2016 shared task. We distinguish three different dimensions: gram-maticality, meaning preservation and simplicity. We show that n-gram-based MT metrics such as BLEU and METEOR correlate the most with human judgment of grammaticality and meaning preservation, whereas simplicity is best evaluated by basic length-based metrics.