mlsanityData to 7 Oct 2026

Measuring and Modifying the Readability of English Texts with GPT-4

arXiv 2410.14028 ↗First posted 17 Oct 2024cs.CLcited by 10

Not on the map. Why?

Abstract (arXiv)

The success of Large Language Models (LLMs) in other domains has raised the question of whether LLMs can reliably assess and manipulate the readability of text. We approach this question empirically. First, using a published corpus of 4,724 English text excerpts, we find that readability estimates produced ``zero-shot'' from GPT-4 Turbo and GPT-4o mini exhibit relatively high correlation with human judgments (r = 0.76 and r = 0.74, respectively), out-performing estimates derived from traditional readability formulas and various psycholinguistic indices. Then, in a pre-registered human experiment (N = 59), we ask whether Turbo can reliably make text easier or harder to read. We find evidence to support this hypothesis, though considerable variance in human judgments remains unexplained. We conclude by discussing the limitations of this approach, including limited scope, as well as the validity of the ``readability'' construct and its dependence on context, audience, and goal.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 13 of its 45 references that are in the map. First posted on arXiv 17 Oct 2024.

Builds on

13 of 45 references.

References · 13

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  3. 2021AI and the Everything in the Whole Wide World Benchmarkcited by 340
  4. 2023Benchmarking Generation and Evaluation Capabilities of Large Language Models for Instruction Controllable Summarizationcited by 69
  5. 2023ChatGPT vs Human-authored Text: Insights into Controllable Text Summarization and Sentence Style Transfercited by 49
  6. 2019Supervised and Unsupervised Neural Approaches to Text Readabilitycited by 45
  7. 2020Linguistic Features for Readability Assessmentcited by 35
  8. 2023BLESS: Benchmarking Large Language Models on Sentence Simplificationcited by 34
  9. 2021Trends, Limitations and Open Challenges in Automatic Readability Assessment Researchcited by 28
  10. 2023Generating Summaries with Controllable Readability Levelscited by 24
  11. 2023Flesch or Fumble? Evaluating Readability Standard Alignment of Instruction-Tuned Language Modelscited by 17
  12. 2018A task in a suit and a tie: paraphrase generation with semantic augmentationcited by 15
1 more
  1. 2022(Psycho-)Linguistic Features Meet Transformer Models for Improved Explainable and Controllable Text Simplificationcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 5; 2026: 5.

10 papers.

  1. 2025Opportunities and Challenges of LLMs in Education: An NLP Perspectivecited by 6
  2. 2026Zero-shot Large Language Models for Automatic Readability Assessmentcited by 3
  3. 2025Rethinking the Role of Text Complexity in Language Model Pretrainingcited by 2
  4. 2025Readability Reconsidered: A Cross-Dataset Analysis of Reference-Free Metricscited by 2
  5. 2025Eye Tracking Based Cognitive Evaluation of Automatic Readability Assessment Methodscited by 1
  6. 2025From Fragments to Facts: A Curriculum-Driven DPO Approach for Generating Hindi News Veracity Explanationscited by 1
  7. 2026DMAP: A Distribution Map for Textcited by 1
  8. 2026Linguistic and Argument Diversity in Synthetic Data for Function-Calling Agentscited by 0
  9. 2026Can LLMs Control Readability? A Multi-Dimensional Evaluation Framework for CEFR-Controlled Arabic Generationcited by 0
  10. 2026Assessing Readability with LLMs: The Role of Reasoning and Few-Shot Promptingcited by 0