# BLESS: Benchmarking Large Language Models on Sentence Simplification

arXiv: https://arxiv.org/abs/2310.15773 · HTML: https://mlsanity.com/paper/2310.15773/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- Builds on 44 of its 80 references that are in the map.
- First posted on arXiv 24 Oct 2023.

## Builds on (most cited first, 30 of 44)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [The Power of Scale for Parameter-Efficient Prompt Tuning](https://mlsanity.com/paper/2104.08691/) (2021)
- [Finetuned Language Models Are Zero-Shot Learners](https://mlsanity.com/paper/2109.01652/) (2021)
- [OPT: Open Pre-trained Transformer Language Models](https://mlsanity.com/paper/2205.01068/) (2022)
- [Scaling Instruction-Finetuned Language Models](https://mlsanity.com/paper/2210.11416/) (2022)
- [The Curious Case of Neural Text Degeneration](https://mlsanity.com/paper/1904.09751/) (2019)
- [Learning to summarize from human feedback](https://mlsanity.com/paper/2009.01325/) (2020)
- [The Pile: An 800GB Dataset of Diverse Text for Language Modeling](https://mlsanity.com/paper/2101.00027/) (2020)
- [G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment](https://mlsanity.com/paper/2303.16634/) (2023)
- [BLOOM: A 176B-Parameter Open-Access Multilingual Language Model](https://mlsanity.com/paper/2211.05100/) (2022)
- [Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity](https://mlsanity.com/paper/2104.08786/) (2021)
- [LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale](https://mlsanity.com/paper/2208.07339/) (2022)
- [GPT-NeoX-20B: An Open-Source Autoregressive Language Model](https://mlsanity.com/paper/2204.06745/) (2022)
- [It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learners](https://mlsanity.com/paper/2009.07118/) (2020)
- [Datasets: A Community Library for Natural Language Processing](https://mlsanity.com/paper/2109.02846/) (2021)
- [The Pushshift Reddit Dataset](https://mlsanity.com/paper/2001.08435/) (2020)
- [OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalization](https://mlsanity.com/paper/2212.12017/) (2022)
- [In-context Examples Selection for Machine Translation](https://mlsanity.com/paper/2212.02437/) (2022)
- [Differentiable Prompt Makes Pre-trained Language Models Better Few-shot Learners](https://mlsanity.com/paper/2108.13161/) (2021)
- [An Analysis of the Automatic Bug Fixing Performance of ChatGPT](https://mlsanity.com/paper/2301.08653/) (2023)
- [Sentence Simplification with Deep Reinforcement Learning](https://mlsanity.com/paper/1703.10931/) (2017)
- [Neural CRF Model for Sentence Alignment in Text Simplification](https://mlsanity.com/paper/2005.02324/) (2020)
- [ASSET: A Dataset for Tuning and Evaluation of Sentence Simplification Models with Multiple Rewriting Transformations](https://mlsanity.com/paper/2005.00481/) (2020)

## Built on by (most cited first, 30 of 35)

- [Turning English-centric LLMs Into Polyglots: How Much Multilinguality Is Needed?](https://mlsanity.com/paper/2312.12683/) (2023)
- [Do Text Simplification Systems Preserve Meaning? A Human Evaluation via Reading Comprehension](https://mlsanity.com/paper/2312.10126/) (2023)
- [Analysing Zero-Shot Readability-Controlled Sentence Simplification](https://mlsanity.com/paper/2409.20246/) (2024)
- [Measuring and Modifying the Readability of English Texts with GPT-4](https://mlsanity.com/paper/2410.14028/) (2024)
- [Attacking Misinformation Detection Using Adversarial Examples Generated by Language Models](https://mlsanity.com/paper/2410.20940/) (2024)
- [LLMs in Education: Novel Perspectives, Challenges, and Opportunities](https://mlsanity.com/paper/2409.11917/) (2024)
- [Opportunities and Challenges of LLMs in Education: An NLP Perspective](https://mlsanity.com/paper/2507.22753/) (2025)
- [Progressive Document-level Text Simplification via Large Language Models](https://mlsanity.com/paper/2501.03857/) (2025)
- [An In-depth Evaluation of Large Language Models in Sentence Simplification with Error-based Human Assessment](https://mlsanity.com/paper/2403.04963/) (2024)
- [Impact of Large Language Model Assistance on Patients Reading Clinical Notes: A Mixed-Methods Study](https://mlsanity.com/paper/2401.09637/) (2024)
- [Digital Comprehensibility Assessment of Simplified Texts among Persons with Intellectual Disabilities](https://mlsanity.com/paper/2402.13094/) (2024)
- [VTechAGP: An Academic-to-General-Audience Text Paraphrase Dataset and Benchmark Models](https://mlsanity.com/paper/2411.04825/) (2024)
- [SiTSE: Sinhala Text Simplification Dataset and Evaluation](https://mlsanity.com/paper/2412.01293/) (2024)
- [Aligning Sentence Simplification with ESL Learner's Proficiency for Language Acquisition](https://mlsanity.com/paper/2502.11457/) (2025)
- [Controlling Reading Ease with Gaze-Guided Text Generation](https://mlsanity.com/paper/2601.17781/) (2026)
- [SpeciaLex: A Benchmark for In-Context Specialized Lexicon Learning](https://mlsanity.com/paper/2407.13297/) (2024)
- [Quebec Automobile Insurance Question-Answering With Retrieval-Augmented Generation](https://mlsanity.com/paper/2410.09623/) (2024)
- [APIO: Automatic Prompt Induction and Optimization for Grammatical Error Correction and Text Simplification](https://mlsanity.com/paper/2508.09378/) (2025)
- [QFrCoLA: a Quebec-French Corpus of Linguistic Acceptability Judgments](https://mlsanity.com/paper/2508.16867/) (2025)
- [Edit-Constrained Decoding for Sentence Simplification](https://mlsanity.com/paper/2409.19247/) (2024)
- [Classic4Children: Adapting Chinese Literary Classics for Children with Large Language Model](https://mlsanity.com/paper/2502.01090/) (2025)
- [Policy-based Sentence Simplification: Replacing Parallel Corpora with LLM-as-a-Judge](https://mlsanity.com/paper/2512.06228/) (2025)
- [Simplify-This: A Comparative Analysis of Prompt-Based and Fine-Tuned LLMs](https://mlsanity.com/paper/2601.05794/) (2026)
- [JUDGEBERT: Assessing Legal Meaning Preservation Between Sentences](https://mlsanity.com/paper/2508.16870/) (2025)
- [Text Adaptation to Plain Language and Easy Read via Automatic Post-Editing Cycles](https://mlsanity.com/paper/2509.11991/) (2025)
- [DETECT: Determining Ease and Textual Clarity of German Text Simplifications](https://mlsanity.com/paper/2510.22212/) (2025)
- [Text Simplification with Sentence Embeddings](https://mlsanity.com/paper/2510.24365/) (2025)
- [Readability Measures and Automatic Text Simplification: In the Search of a Construct](https://mlsanity.com/paper/2511.09536/) (2025)
- [Evaluating Small Decoder-Only Language Models for Grammar Correction and Text Simplification](https://mlsanity.com/paper/2601.03874/) (2026)
- [OasisSimp: An Open-source Asian-English Sentence Simplification Dataset](https://mlsanity.com/paper/2603.14111/) (2026)

## Abstract (arXiv)

We present BLESS, a comprehensive performance benchmark of the most recent state-of-the-art large language models (LLMs) on the task of text simplification (TS). We examine how well off-the-shelf LLMs can solve this challenging task, assessing a total of 44 models, differing in size, architecture, pre-training methods, and accessibility, on three test sets from different domains (Wikipedia, news, and medical) under a few-shot setting. Our analysis considers a suite of automatic metrics as well as a large-scale quantitative investigation into the types of common edit operations performed by the different models. Furthermore, we perform a manual qualitative analysis on a subset of model outputs to better gauge the quality of the generated simplifications. Our evaluation indicates that the best LLMs, despite not being trained on TS, perform comparably with state-of-the-art TS baselines. Additionally, we find that certain LLMs demonstrate a greater range and diversity of edit operations. Our performance benchmark will be available as a resource for the development of future TS methods and evaluation metrics.