# Improving Human Sequential Decision-Making with Reinforcement Learning

arXiv: https://arxiv.org/abs/2108.08454 · HTML: https://mlsanity.com/paper/2108.08454/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 76 references that are in the map.
- First posted on arXiv 19 Aug 2021.

## Builds on (most cited first, 14 of 14)

- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Towards A Rigorous Science of Interpretable Machine Learning](https://mlsanity.com/paper/1702.08608/) (2017)
- [Programmatically Interpretable Reinforcement Learning](https://mlsanity.com/paper/1804.02477/) (2018)
- [Verifiable Reinforcement Learning via Policy Extraction](https://mlsanity.com/paper/1805.08328/) (2018)
- [On Human Predictions with Explanations and Predictions of Machine Learning Models: A Case Study on Deception Detection](https://mlsanity.com/paper/1811.07901/) (2018)
- [Explainable Reinforcement Learning: A Survey](https://mlsanity.com/paper/2005.06247/) (2020)
- [Interpretable classifiers using rules and Bayesian analysis: Building a better stroke prediction model](https://mlsanity.com/paper/1511.01644/) (2015)
- [Aligning Superhuman AI with Human Behavior: Chess as a Model System](https://mlsanity.com/paper/2006.01855/) (2020)
- [Do Explanations make VQA Models more Predictable to a Human?](https://mlsanity.com/paper/1810.12366/) (2018)
- [Human-in-the-Loop Interpretability Prior](https://mlsanity.com/paper/1805.11571/) (2018)
- [It Takes Two to Tango: Towards Theory of AI's Mind](https://mlsanity.com/paper/1704.00717/) (2017)
- [Measuring the Completeness of Theories](https://mlsanity.com/paper/1910.07022/) (2019)
- [Improving Human-Machine Cooperative Visual Search With Soft Highlighting](https://mlsanity.com/paper/1612.08117/) (2016)

## Built on by (most cited first, 12 of 12)

- [Algorithmic Assistance with Recommendation-Dependent Preferences](https://mlsanity.com/paper/2208.07626/) (2022)
- [AI Agents for Inventory Control: Human-LLM-OR Complementarity](https://mlsanity.com/paper/2602.12631/) (2026)
- [On the Fragility of AI Agent Collusion](https://mlsanity.com/paper/2603.20281/) (2026)
- [A Q-learning Approach for Adherence-Aware Recommendations](https://mlsanity.com/paper/2309.06519/) (2023)
- [Expanding Knowledge Graphs with Humans in the Loop](https://mlsanity.com/paper/2212.05189/) (2022)
- [Narrowing Action Choices with AI Improves Human Sequential Decisions](https://mlsanity.com/paper/2510.16097/) (2025)
- [HR-Bandit: Human-AI Collaborated Linear Recourse Bandit](https://mlsanity.com/paper/2410.14640/) (2024)
- [LLM-Enhanced Reinforcement Learning for Time Series Anomaly Detection](https://mlsanity.com/paper/2601.02511/) (2026)
- [Overcoming Temptation: Incentive Design For Intertemporal Choice](https://mlsanity.com/paper/2203.05782/) (2022)
- [Strategic Advice in the Age of Personal AI](https://mlsanity.com/paper/2603.02055/) (2026)
- [Improving Human Performance with Value-Aware Interventions: A Case Study in Chess](https://mlsanity.com/paper/2604.14465/) (2026)
- [Algorithmic Feature Highlighting for Human-AI Decision-Making](https://mlsanity.com/paper/2604.22236/) (2026)

## Abstract (arXiv)

Workers spend a significant amount of time learning how to make good decisions. Evaluating the efficacy of a given decision, however, can be complicated -- e.g., decision outcomes are often long-term and relate to the original decision in complex ways. Surprisingly, even though learning good decision-making strategies is difficult, they can often be expressed in simple and concise forms. Focusing on sequential decision-making, we design a novel machine learning algorithm that is capable of extracting "best practices" from trace data and conveying its insights to humans in the form of interpretable "tips". Our algorithm selects the tip that best bridges the gap between the actions taken by human workers and those taken by the optimal policy in a way that accounts for which actions are consequential for achieving higher performance. We evaluate our approach through a series of randomized controlled experiments where participants manage a virtual kitchen. Our experiments show that the tips generated by our algorithm can significantly improve human performance relative to intuitive baselines. In addition, we discuss a number of empirical insights that can help inform the design of algorithms intended for human-AI interfaces. For instance, we find evidence that participants do not simply blindly follow our tips; instead, they combine them with their own experience to discover additional strategies for improving performance.