# Beyond Flesch-Kincaid: Prompt-based Metrics Improve Difficulty Classification of Educational Texts

arXiv: https://arxiv.org/abs/2405.09482 · HTML: https://mlsanity.com/paper/2405.09482/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 34 references that are in the map.
- First posted on arXiv 15 May 2024.

## Builds on (most cited first, 8 of 8)

- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [Scaling Instruction-Finetuned Language Models](https://mlsanity.com/paper/2210.11416/) (2022)
- [Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering](https://mlsanity.com/paper/2209.09513/) (2022)
- [Flesch or Fumble? Evaluating Readability Standard Alignment of Instruction-Tuned Language Models](https://mlsanity.com/paper/2309.05454/) (2023)
- [Know Your Audience: Do LLMs Adapt to Different Age and Education Levels?](https://mlsanity.com/paper/2312.02065/) (2023)
- [Simple or Complex? Complexity-Controllable Question Generation with Soft Templates and Deep Mixture of Experts Model](https://mlsanity.com/paper/2110.06560/) (2021)
- [Do LLMs Implicitly Determine the Suitable Text Difficulty for Users?](https://mlsanity.com/paper/2402.14453/) (2024)

## Built on by (most cited first, 9 of 9)

- [Biased Tales: Cultural and Topic Bias in Generating Children's Stories](https://mlsanity.com/paper/2509.07908/) (2025)
- [Multilingual Performance Biases of Large Language Models in Education](https://mlsanity.com/paper/2504.17720/) (2025)
- [Alignment Drift in CEFR-prompted LLMs for Interactive Spanish Tutoring](https://mlsanity.com/paper/2505.08351/) (2025)
- [Opportunities and Challenges of LLMs in Education: An NLP Perspective](https://mlsanity.com/paper/2507.22753/) (2025)
- [PATS: Personality-Aware Teaching Strategies with Large Language Model Tutors](https://mlsanity.com/paper/2601.08402/) (2026)
- [Rethinking the Role of Text Complexity in Language Model Pretraining](https://mlsanity.com/paper/2509.16551/) (2025)
- [Readability Reconsidered: A Cross-Dataset Analysis of Reference-Free Metrics](https://mlsanity.com/paper/2510.15345/) (2025)
- [Knowledge without Wisdom: Measuring Misalignment between LLMs and Intended Impact](https://mlsanity.com/paper/2603.00883/) (2026)
- [Incentives Of EdTech: A Systematic Review Of EduNLP Research](https://mlsanity.com/paper/2606.13691/) (2026)

## Abstract (arXiv)

Using large language models (LLMs) for educational applications like dialogue-based teaching is a hot topic. Effective teaching, however, requires teachers to adapt the difficulty of content and explanations to the education level of their students. Even the best LLMs today struggle to do this well. If we want to improve LLMs on this adaptation task, we need to be able to measure adaptation success reliably. However, current Static metrics for text difficulty, like the Flesch-Kincaid Reading Ease score, are known to be crude and brittle. We, therefore, introduce and evaluate a new set of Prompt-based metrics for text difficulty. Based on a user study, we create Prompt-based metrics as inputs for LLMs. They leverage LLM's general language understanding capabilities to capture more abstract and complex features than Static metrics. Regression experiments show that adding our Prompt-based metrics significantly improves text difficulty classification over Static metrics alone. Our results demonstrate the promise of using LLMs to evaluate text adaptation to different education levels.