# Are Machine Rationales (Not) Useful to Humans? Measuring and Improving Human Utility of Free-Text Rationales

arXiv: https://arxiv.org/abs/2305.07095 · HTML: https://mlsanity.com/paper/2305.07095/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 61 references that are in the map.
- First posted on arXiv 11 May 2023.

## Builds on (most cited first, 30 of 38)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answering](https://mlsanity.com/paper/1809.02789/) (2018)
- [Decision Transformer: Reinforcement Learning via Sequence Modeling](https://mlsanity.com/paper/2106.01345/) (2021)
- [Towards A Rigorous Science of Interpretable Machine Learning](https://mlsanity.com/paper/1702.08608/) (2017)
- [Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategies](https://mlsanity.com/paper/2101.02235/) (2021)
- [Offline Reinforcement Learning as One Big Sequence Modeling Problem](https://mlsanity.com/paper/2106.02039/) (2021)
- [UnifiedQA: Crossing Format Boundaries With a Single QA System](https://mlsanity.com/paper/2005.00700/) (2020)
- [e-SNLI: Natural Language Inference with Natural Language Explanations](https://mlsanity.com/paper/1812.01193/) (2018)
- [Self-critiquing models for assisting human evaluators](https://mlsanity.com/paper/2206.05802/) (2022)
- [Explain Yourself! Leveraging Language Models for Commonsense Reasoning](https://mlsanity.com/paper/1906.02361/) (2019)
- [Can language models learn from explanations in context?](https://mlsanity.com/paper/2204.02329/) (2022)
- [Quark: Controllable Text Generation with Reinforced Unlearning](https://mlsanity.com/paper/2205.13636/) (2022)
- [WANLI: Worker and AI Collaboration for Natural Language Inference Dataset Creation](https://mlsanity.com/paper/2201.05955/) (2022)
- [ROSCOE: A Suite of Metrics for Scoring Step-by-Step Reasoning](https://mlsanity.com/paper/2212.07919/) (2022)
- [Measuring Association Between Labels and Free-Text Rationales](https://mlsanity.com/paper/2010.12762/) (2020)
- [WT5?! Training Text-to-Text Models to Explain their Predictions](https://mlsanity.com/paper/2004.14546/) (2020)
- [Teach Me to Explain: A Review of Datasets for Explainable Natural Language Processing](https://mlsanity.com/paper/2102.12060/) (2021)
- [Reframing Human-AI Collaboration for Generating Free-Text Explanations](https://mlsanity.com/paper/2112.08674/) (2021)
- [CommonsenseQA 2.0: Exposing the Limits of AI through Gamification](https://mlsanity.com/paper/2201.05320/) (2022)
- [Leakage-Adjusted Simulatability: Can Models Generate Non-Trivial Explanations of Their Behavior in Natural Language?](https://mlsanity.com/paper/2010.04119/) (2020)
- [Few-Shot Self-Rationalization with Natural Language Prompts](https://mlsanity.com/paper/2111.08284/) (2021)
- [UnifiedQA-v2: Stronger Generalization via Broader Cross-Format Training](https://mlsanity.com/paper/2202.12359/) (2022)
- [QED: A Framework and Dataset for Explanations in Question Answering](https://mlsanity.com/paper/2009.06354/) (2020)
- [What can AI do for me: Evaluating Machine Learning Interpretations in Cooperative Play](https://mlsanity.com/paper/1810.09648/) (2018)
- [Prompting Contrastive Explanations for Commonsense Reasoning Tasks](https://mlsanity.com/paper/2106.06823/) (2021)
- [Rationalization: A Neural Machine Translation Approach to Generating Natural Language Explanations](https://mlsanity.com/paper/1702.07826/) (2017)

## Built on by (most cited first, 30 of 37)

- [To CoT or not to CoT? Chain-of-thought helps mainly on math and symbolic reasoning](https://mlsanity.com/paper/2409.12183/) (2024)
- [CoT-Valve: Length-Compressible Chain-of-Thought Tuning](https://mlsanity.com/paper/2502.09601/) (2025)
- [Do Models Explain Themselves? Counterfactual Simulatability of Natural Language Explanations](https://mlsanity.com/paper/2307.08678/) (2023)
- [When Thinking Fails: The Pitfalls of Reasoning for Instruction-Following in LLMs](https://mlsanity.com/paper/2505.11423/) (2025)
- [Large Language Models Help Humans Verify Truthfulness -- Except When They Are Convincingly Wrong](https://mlsanity.com/paper/2310.12558/) (2023)
- [Measuring Chain of Thought Faithfulness by Unlearning Reasoning Steps](https://mlsanity.com/paper/2502.14829/) (2025)
- [Think or Not Think: A Study of Explicit Thinking in Rule-Based Visual Reinforcement Fine-Tuning](https://mlsanity.com/paper/2503.16188/) (2025)
- [LLM-Generated Black-box Explanations Can Be Adversarially Helpful](https://mlsanity.com/paper/2405.06800/) (2024)
- [From Calibration to Collaboration: LLM Uncertainty Quantification Should Be More Human-Centered](https://mlsanity.com/paper/2506.07461/) (2025)
- [Effective Human-AI Teams via Learned Natural Language Rules and Onboarding](https://mlsanity.com/paper/2311.01007/) (2023)
- [Position: Towards Bidirectional Human-AI Alignment](https://mlsanity.com/paper/2406.09264/) (2024)
- [xTower: A Multilingual LLM for Explaining and Correcting Translation Errors](https://mlsanity.com/paper/2406.19482/) (2024)
- [Can Language Models Teach Weaker Agents? Teacher Explanations Improve Students via Personalization](https://mlsanity.com/paper/2306.09299/) (2023)
- [Modulating Language Model Experiences through Frictions](https://mlsanity.com/paper/2407.12804/) (2024)
- [Can Small Language Models Help Large Language Models Reason Better?: LM-Guided Chain-of-Thought](https://mlsanity.com/paper/2404.03414/) (2024)
- [Characterizing Large Language Models as Rationalizers of Knowledge-intensive Tasks](https://mlsanity.com/paper/2311.05085/) (2023)
- [On Evaluating Explanation Utility for Human-AI Decision Making in NLP](https://mlsanity.com/paper/2407.03545/) (2024)
- [Improving Language Model Personas via Rationalization with Psychological Scaffolds](https://mlsanity.com/paper/2504.17993/) (2025)
- [The Quest for Efficient Reasoning: A Data-Centric Benchmark to CoT Distillation](https://mlsanity.com/paper/2505.18759/) (2025)
- [Fact :Teaching MLLMs with Faithful, Concise and Transferable Rationales](https://mlsanity.com/paper/2404.11129/) (2024)
- [Persuasiveness of Generated Free-Text Rationales in Subjective Decisions: A Case Study on Pairwise Argument Ranking](https://mlsanity.com/paper/2406.13905/) (2024)
- [Defining Knowledge: Bridging Epistemology and Large Language Models](https://mlsanity.com/paper/2410.02499/) (2024)
- [Adjust for Trust: Mitigating Trust-Induced Inappropriate Reliance on AI Assistance](https://mlsanity.com/paper/2502.13321/) (2025)
- [Explanation-aware Soft Ensemble Empowers Large Language Model In-context Learning](https://mlsanity.com/paper/2311.07099/) (2023)
- [Digital Socrates: Evaluating LLMs through Explanation Critiques](https://mlsanity.com/paper/2311.09613/) (2023)
- [Free-text Rationale Generation under Readability Level Control](https://mlsanity.com/paper/2407.01384/) (2024)
- [ELI-Why: Evaluating the Pedagogical Utility of Language Model Explanations](https://mlsanity.com/paper/2506.14200/) (2025)
- [ACORN: Aspect-wise Commonsense Reasoning Explanation Evaluation](https://mlsanity.com/paper/2405.04818/) (2024)
- [CAVE: Controllable Authorship Verification Explanations](https://mlsanity.com/paper/2406.16672/) (2024)
- [Rethinking Human Preference Evaluation of LLM Rationales](https://mlsanity.com/paper/2509.11026/) (2025)

## Abstract (arXiv)

Among the remarkable emergent capabilities of large language models (LMs) is free-text rationalization; beyond a certain scale, large LMs are capable of generating seemingly useful rationalizations, which in turn, can dramatically enhance their performances on leaderboards. This phenomenon raises a question: can machine generated rationales also be useful for humans, especially when lay humans try to answer questions based on those machine rationales? We observe that human utility of existing rationales is far from satisfactory, and expensive to estimate with human studies. Existing metrics like task performance of the LM generating the rationales, or similarity between generated and gold rationales are not good indicators of their human utility. While we observe that certain properties of rationales like conciseness and novelty are correlated with their human utility, estimating them without human involvement is challenging. We show that, by estimating a rationale's helpfulness in answering similar unseen instances, we can measure its human utility to a better extent. We also translate this finding into an automated score, GEN-U, that we propose, which can help improve LMs' ability to generate rationales with better human utility, while maintaining most of its task performance. Lastly, we release all code and collected data with this project.