# CEScore: Simple and Efficient Confidence Estimation Model for Evaluating Split and Rephrase

arXiv: https://arxiv.org/abs/2312.01356 · HTML: https://mlsanity.com/paper/2312.01356/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 10 of its 43 references that are in the map.
- First posted on arXiv 3 Dec 2023.

## Builds on (most cited first, 10 of 10)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [BLEU is Not Suitable for the Evaluation of Text Simplification](https://mlsanity.com/paper/1810.05995/) (2018)
- [Semantic Structural Evaluation for Text Simplification](https://mlsanity.com/paper/1810.05022/) (2018)
- [Split and Rephrase](https://mlsanity.com/paper/1707.06971/) (2017)
- [Simple and Effective Text Simplification Using Semantic and Neural Methods](https://mlsanity.com/paper/1810.05104/) (2018)
- [Split and Rephrase: Better Evaluation and a Stronger Baseline](https://mlsanity.com/paper/1805.01035/) (2018)
- [Transforming Complex Sentences into a Semantic Hierarchy](https://mlsanity.com/paper/1906.01038/) (2019)
- [A Transition-Based Directed Acyclic Graph Parser for UCCA](https://mlsanity.com/paper/1704.00552/) (2017)
- [Reference-less Quality Estimation of Text Simplification Systems](https://mlsanity.com/paper/1901.10746/) (2019)

## Abstract (arXiv)

The split and rephrase (SR) task aims to divide a long, complex sentence into a set of shorter, simpler sentences that convey the same meaning. This challenging problem in NLP has gained increased attention recently because of its benefits as a pre-processing step in other NLP tasks. Evaluating quality of SR is challenging, as there no automatic metric fit to evaluate this task. In this work, we introduce CEScore, as novel statistical model to automatically evaluate SR task. By mimicking the way humans evaluate SR, CEScore provides 4 metrics (Sscore, Gscore, Mscore, and CEscore) to assess simplicity, grammaticality, meaning preservation, and overall quality, respectively. In experiments with 26 models, CEScore correlates strongly with human evaluations, achieving 0.98 in Spearman correlations at model-level. This underscores the potential of CEScore as a simple and effective metric for assessing the overall quality of SR models.