# Calibrating LLM Explanations to Human Decisions with Reinforcement Learning

arXiv: https://arxiv.org/abs/2610.17306 · HTML: https://mlsanity.com/paper/2610.17306/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 21 of its 29 references that are in the map.
- First posted on arXiv 12 Oct 2026.

## Builds on (most cited first, 21 of 21)

- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Deep reinforcement learning from human preferences](https://mlsanity.com/paper/1706.03741/) (2017)
- [Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting](https://mlsanity.com/paper/2305.04388/) (2023)
- [Measuring Faithfulness in Chain-of-Thought Reasoning](https://mlsanity.com/paper/2307.13702/) (2023)
- [RLAIF vs. RLHF: Scaling Reinforcement Learning from Human Feedback with AI Feedback](https://mlsanity.com/paper/2309.00267/) (2023)
- [Proxy Tasks and Subjective Measures Can Be Misleading in Evaluating Explainable AI Systems](https://mlsanity.com/paper/2001.08298/) (2020)
- [MGTBench: Benchmarking Machine-Generated Text Detection](https://mlsanity.com/paper/2303.14822/) (2023)
- [People who frequently use ChatGPT for writing tasks are accurate and robust detectors of AI-generated text](https://mlsanity.com/paper/2501.15654/) (2025)
- [How to Distinguish AI-Generated Images from Authentic Photographs](https://mlsanity.com/paper/2406.08651/) (2024)
- [Utilizing Human Behavior Modeling to Manipulate Explanations in AI-Assisted Decision Making: The Good, the Bad, and the Scary](https://mlsanity.com/paper/2411.10461/) (2024)
- [Beyond Tools: Understanding How Heavy Users Integrate LLMs into Everyday Tasks and Decision-Making](https://mlsanity.com/paper/2502.15395/) (2025)
- [AI Agents Push Humans Out of the Loop](https://mlsanity.com/paper/2608.23642/) (2026)
- [User Decision Guidance with Selective Explanation Presentation from Explainable-AI](https://mlsanity.com/paper/2402.18016/) (2024)
- [Arguments that Alter Minds: LLM Rationales Sway Human (and LLM) Notions of Plausibility](https://mlsanity.com/paper/2510.08091/) (2025)
- [Aligning Deep Implicit Preferences by Learning to Reason Defensively](https://mlsanity.com/paper/2510.11194/) (2025)
- [ComplLLM: Fine-tuning LLMs to Discover Complementary Signals for Decision-making](https://mlsanity.com/paper/2602.19458/) (2026)
- [Plausibility as Failure: How LLMs and Humans Co-Construct Epistemic Error](https://mlsanity.com/paper/2512.16750/) (2025)
- ["The explanation makes sense": An Empirical Study on LLM Performance in News Classification and its Influence on Judgment in Human-AI Collaborative Annotation](https://mlsanity.com/paper/2602.19690/) (2026)

## Abstract (arXiv)

Explanations of LLM outputs are considered an important way to make model behavior legible, helping human users make good decisions on downstream tasks. However, default training approaches for improving explanations target model task performance or human preference rather than human decision quality. To bridge this gap, we introduce HD-RL, a reinforcement learning framework for optimizing LLM explanations for human decision making. Our approach calibrates a human decision simulator on observed human decisions, applying a proxy reward model to optimize the LLM to generate explanations that help people to make good decisions. We apply this framework to AI-generated text detection. In our initial data collection phase, we observed that while LLMs possess strong autonomous detection capabilities, the explanations they generate for users yield near-random human accuracy (51.3%). By applying the HD-RL framework to the open-source Qwen 3.6 model, we generate optimized explanations that improve human performance, achieving a GLMM-estimated human accuracy of 63.7% in a randomized between-subjects experiment with repeated measures and outperforming RLVR, RLAIF, and other baselines by over 8%.