# Free-text Rationale Generation under Readability Level Control

arXiv: https://arxiv.org/abs/2407.01384 · HTML: https://mlsanity.com/paper/2407.01384/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 58 references that are in the map.
- First posted on arXiv 1 Jul 2024.

## Builds on (most cited first, 29 of 29)

- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment](https://mlsanity.com/paper/2303.16634/) (2023)
- [Length-Controlled AlpacaEval: A Simple Way to Debias Automatic Evaluators](https://mlsanity.com/paper/2404.04475/) (2024)
- [Plug and Play Language Models: A Simple Approach to Controlled Text Generation](https://mlsanity.com/paper/1912.02164/) (2019)
- [LLM Evaluators Recognize and Favor Their Own Generations](https://mlsanity.com/paper/2404.13076/) (2024)
- [Is ChatGPT a Good NLG Evaluator? A Preliminary Study](https://mlsanity.com/paper/2303.04048/) (2023)
- [HateXplain: A Benchmark Dataset for Explainable Hate Speech Detection](https://mlsanity.com/paper/2012.10289/) (2020)
- [A Survey on Neural Network Interpretability](https://mlsanity.com/paper/2012.14261/) (2020)
- [OpenChat: Advancing Open-source Language Models with Mixed-Quality Data](https://mlsanity.com/paper/2309.11235/) (2023)
- [The Unreliability of Explanations in Few-shot Prompting for Textual Reasoning](https://mlsanity.com/paper/2205.03401/) (2022)
- [A Theory of Usable Information Under Computational Constraints](https://mlsanity.com/paper/2002.10689/) (2020)
- [Measuring Association Between Labels and Free-Text Rationales](https://mlsanity.com/paper/2010.12762/) (2020)
- [Faithfulness Tests for Natural Language Explanations](https://mlsanity.com/paper/2305.18029/) (2023)
- [Leakage-Adjusted Simulatability: Can Models Generate Non-Trivial Explanations of Their Behavior in Natural Language?](https://mlsanity.com/paper/2010.04119/) (2020)
- [Can Large Language Models Explain Themselves? A Study of LLM-Generated Self-Explanations](https://mlsanity.com/paper/2310.11207/) (2023)
- [Automated Rationale Generation: A Technique for Explainable AI and its Effects on Human Perceptions](https://mlsanity.com/paper/1901.03729/) (2019)
- [ReCEval: Evaluating Reasoning Chains via Correctness and Informativeness](https://mlsanity.com/paper/2304.10703/) (2023)
- [Conditional probing: measuring usable information beyond a baseline](https://mlsanity.com/paper/2109.09234/) (2021)
- [Visualizing the Feature Importance for Black Box Models](https://mlsanity.com/paper/1804.06620/) (2018)
- [Are Machine Rationales (Not) Useful to Humans? Measuring and Improving Human Utility of Free-Text Rationales](https://mlsanity.com/paper/2305.07095/) (2023)
- [HealthFC: Verifying Health Claims with Evidence-Based Medical Fact-Checking](https://mlsanity.com/paper/2309.08503/) (2023)
- [Elaborative Simplification: Content Addition and Explanation Generation in Text Simplification](https://mlsanity.com/paper/2010.10035/) (2020)
- [Generating Summaries with Controllable Readability Levels](https://mlsanity.com/paper/2310.10623/) (2023)
- [Know Your Audience: Do LLMs Adapt to Different Age and Education Levels?](https://mlsanity.com/paper/2312.02065/) (2023)
- [Generating Educational Materials with Different Levels of Readability using LLMs](https://mlsanity.com/paper/2406.12787/) (2024)
- [Uniform Complexity for Text Generation](https://mlsanity.com/paper/2204.05185/) (2022)
- [Do LLMs Implicitly Determine the Suitable Text Difficulty for Users?](https://mlsanity.com/paper/2402.14453/) (2024)
- [RORA: Robust Free-Text Rationale Evaluation](https://mlsanity.com/paper/2402.18678/) (2024)

## Built on by (most cited first, 4 of 4)

- [DOGe: Defensive Output Generation for LLM Protection Against Knowledge Distillation](https://mlsanity.com/paper/2505.19504/) (2025)
- [Can Large Language Models Still Explain Themselves? Investigating the Impact of Quantization on Self-Explanations](https://mlsanity.com/paper/2601.00282/) (2026)
- [ELI-Why: Evaluating the Pedagogical Utility of Language Model Explanations](https://mlsanity.com/paper/2506.14200/) (2025)
- [Classroom AI: Large Language Models as Grade-Specific Teachers](https://mlsanity.com/paper/2601.06225/) (2026)

## Abstract (arXiv)

Free-text rationales justify model decisions in natural language and thus become likable and accessible among approaches to explanation across many tasks. However, their effectiveness can be hindered by misinterpretation and hallucination. As a perturbation test, we investigate how large language models (LLMs) perform rationale generation under the effects of readability level control, i.e., being prompted for an explanation targeting a specific expertise level, such as sixth grade or college. We find that explanations are adaptable to such instruction, though the observed distinction between readability levels does not fully match the defined complexity scores according to traditional readability metrics. Furthermore, the generated rationales tend to feature medium level complexity, which correlates with the measured quality using automatic metrics. Finally, our human annotators confirm a generally satisfactory impression on rationales at all readability levels, with high-school-level readability being most commonly perceived and favored.