# A Q-learning Approach for Adherence-Aware Recommendations

arXiv: https://arxiv.org/abs/2309.06519 · HTML: https://mlsanity.com/paper/2309.06519/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 26 references that are in the map.
- First posted on arXiv 12 Sep 2023.

## Builds on (most cited first, 8 of 8)

- [Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learning](https://mlsanity.com/paper/2108.06266/) (2021)
- [The Best Decisions Are Not the Best Advice: Making Adherence-Aware Recommendations](https://mlsanity.com/paper/2209.01874/) (2022)
- [Improving Human Sequential Decision-Making with Reinforcement Learning](https://mlsanity.com/paper/2108.08454/) (2021)
- [On Team Decision Problems with Nonclassical Information Structures](https://mlsanity.com/paper/2101.10992/) (2021)
- [Separation of Learning and Control for Cyber-Physical Systems](https://mlsanity.com/paper/2107.06379/) (2021)
- [Connected and Automated Vehicles in Mixed-Traffic: Learning Human Driver Behavior for Effective On-Ramp Merging](https://mlsanity.com/paper/2304.00397/) (2023)
- [Optimal Time Trajectory and Coordination for Connected and Automated Vehicles](https://mlsanity.com/paper/2003.12183/) (2020)
- [Worst-Case Control and Learning Using Partial Observations Over an Infinite Time-Horizon](https://mlsanity.com/paper/2303.16321/) (2023)

## Built on by (most cited first, 4 of 4)

- [A Framework for Effective AI Recommendations in Cyber-Physical-Human Systems](https://mlsanity.com/paper/2403.05715/) (2024)
- [AI Recommendation Systems for Lane-Changing Using Adherence-Aware Reinforcement Learning](https://mlsanity.com/paper/2504.20187/) (2025)
- [Off-Policy Evaluation for Sequential Persuasion Process with Unobserved Confounding](https://mlsanity.com/paper/2504.01211/) (2025)
- [Reward Design for Justifiable Sequential Decision-Making](https://mlsanity.com/paper/2402.15826/) (2024)

## Abstract (arXiv)

In many real-world scenarios involving high-stakes and safety implications, a human decision-maker (HDM) may receive recommendations from an artificial intelligence while holding the ultimate responsibility of making decisions. In this letter, we develop an "adherence-aware Q-learning" algorithm to address this problem. The algorithm learns the "adherence level" that captures the frequency with which an HDM follows the recommended actions and derives the best recommendation policy in real time. We prove the convergence of the proposed Q-learning algorithm to the optimal value and evaluate its performance across various scenarios.