# Semantic Structural Evaluation for Text Simplification

arXiv: https://arxiv.org/abs/1810.05022 · HTML: https://mlsanity.com/paper/1810.05022/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Oct 2018.

## Built on by (most cited first, 30 of 30)

- [LENS: A Learnable Evaluation Metric for Text Simplification](https://mlsanity.com/paper/2212.09739/) (2022)
- [EditEval: An Instruction-Based Benchmark for Text Improvements](https://mlsanity.com/paper/2209.13331/) (2022)
- [Document-Level Text Simplification: Dataset, Criteria and Baseline](https://mlsanity.com/paper/2110.05071/) (2021)
- [Themis: A Reference-free NLG Evaluation Language Model with Flexibility and Interpretability](https://mlsanity.com/paper/2406.18365/) (2024)
- [Dancing Between Success and Failure: Edit-level Simplification Evaluation using SALSA](https://mlsanity.com/paper/2305.14458/) (2023)
- [DEPLAIN: A German Parallel Corpus with Intralingual Translations into Plain Language for Sentence and Document Simplification](https://mlsanity.com/paper/2305.18939/) (2023)
- [Simplicity Level Estimate (SLE): A Learned Reference-Less Metric for Sentence Simplification](https://mlsanity.com/paper/2310.08170/) (2023)
- [GRS: Combining Generation and Revision in Unsupervised Sentence Simplification](https://mlsanity.com/paper/2203.09742/) (2022)
- [Opportunities and Challenges of LLMs in Education: An NLP Perspective](https://mlsanity.com/paper/2507.22753/) (2025)
- [Semantics-aware Attention Improves Neural Machine Translation](https://mlsanity.com/paper/2110.06920/) (2021)
- [Rethinking Semantic Parsing for Large Language Models: Enhancing LLM Performance with Semantic Hints](https://mlsanity.com/paper/2409.14469/) (2024)
- [Revision for Concision: A Constrained Paraphrase Generation Task](https://mlsanity.com/paper/2210.14257/) (2022)
- [A New Dataset and Empirical Study for Sentence Simplification in Chinese](https://mlsanity.com/paper/2306.04188/) (2023)
- [Medical Text Simplification: Optimizing for Readability with Unlikelihood Training and Reranked Beam Search Decoding](https://mlsanity.com/paper/2310.11191/) (2023)
- [REFeREE: A REference-FREE Model-Based Metric for Text Simplification](https://mlsanity.com/paper/2403.17640/) (2024)
- [Aligning Sentence Simplification with ESL Learner's Proficiency for Language Acquisition](https://mlsanity.com/paper/2502.11457/) (2025)
- [Near-Negative Distinction: Giving a Second Life to Human Evaluation Datasets](https://mlsanity.com/paper/2205.06871/) (2022)
- [The Fewer Splits are Better: Deconstructing Readability in Sentence Splitting](https://mlsanity.com/paper/2302.00937/) (2023)
- [Discourse-Aware Text Simplification: From Complex Sentences to Linked Propositions](https://mlsanity.com/paper/2308.00425/) (2023)
- [A User-Centered Evaluation of Spanish Text Simplification](https://mlsanity.com/paper/2308.07556/) (2023)
- [Effects of Added Emphasis and Pause in Audio Delivery of Health Information](https://mlsanity.com/paper/2404.19119/) (2024)
- [Readability-guided Idiom-aware Sentence Simplification (RISS) for Chinese](https://mlsanity.com/paper/2406.02974/) (2024)
- [A Human-in/on-the-Loop Framework for Accessible Text Generation](https://mlsanity.com/paper/2603.18879/) (2026)
- [Automatic Text Simplification of News Articles in the Context of Public Broadcasting](https://mlsanity.com/paper/2212.13317/) (2022)
- [LC-Score: Reference-less estimation of Text Comprehension Difficulty](https://mlsanity.com/paper/2310.02754/) (2023)
- [CEScore: Simple and Efficient Confidence Estimation Model for Evaluating Split and Rephrase](https://mlsanity.com/paper/2312.01356/) (2023)
- [EASSE-DE: Easier Automatic Sentence Simplification Evaluation for German](https://mlsanity.com/paper/2404.03563/) (2024)
- [Automated Feedback Loops to Protect Text Simplification with Generative AI from Information Loss](https://mlsanity.com/paper/2505.16172/) (2025)
- [DETECT: Determining Ease and Textual Clarity of German Text Simplifications](https://mlsanity.com/paper/2510.22212/) (2025)
- [Profiling German Text Simplification with Interpretable Model-Fingerprints](https://mlsanity.com/paper/2601.13050/) (2026)

## Abstract (arXiv)

Current measures for evaluating text simplification systems focus on evaluating lexical text aspects, neglecting its structural aspects. In this paper we propose the first measure to address structural aspects of text simplification, called SAMSA. It leverages recent advances in semantic parsing to assess simplification quality by decomposing the input based on its semantic structure and comparing it to the output. SAMSA provides a reference-less automatic evaluation procedure, avoiding the problems that reference-based methods face due to the vast space of valid simplifications for a given sentence. Our human evaluation experiments show both SAMSA's substantial correlation with human judgments, as well as the deficiency of existing reference-based measures in evaluating structural simplification.