# Towards Optimizing Human-Centric Objectives in AI-Assisted Decision-Making With Offline Reinforcement Learning arXiv: https://arxiv.org/abs/2403.05911 ยท HTML: https://mlsanity.com/paper/2403.05911/ ## Key facts - Cited by 13 papers in the map (data through 14 Oct 2026). - Placed in Offline RL algorithms, one of 13 directions in Offline RL and imitation learning from data. - Builds on 20 of its 82 references that are in the map. - First posted on arXiv 9 Mar 2024. ## Placed in - Offline RL and imitation learning from data / [Offline RL algorithms](https://mlsanity.com/offlinerl/offline-rl-algorithms/) ## Builds on (most cited first, 20 of 20) - [Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems](https://mlsanity.com/paper/2005.01643/) (2020) - [Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance](https://mlsanity.com/paper/2006.14779/) (2020) - [Effect of Confidence and Explanation on Accuracy and Trust Calibration in AI-Assisted Decision Making](https://mlsanity.com/paper/2001.02114/) (2020) - [Manipulating and Measuring Model Interpretability](https://mlsanity.com/paper/1802.07810/) (2018) - [Proxy Tasks and Subjective Measures Can Be Misleading in Evaluating Explainable AI Systems](https://mlsanity.com/paper/2001.08298/) (2020) - [Understanding the Role of Human Intuition on Reliance in Human-AI Decision-Making with Explanations](https://mlsanity.com/paper/2301.07255/) (2023) - [Who Should I Trust: AI or Myself? Leveraging Human and AI Correctness Likelihood to Promote Appropriate Trust in AI-Assisted Decision-Making](https://mlsanity.com/paper/2301.05809/) (2023) - [Explainable AI is Dead, Long Live Explainable AI! Hypothesis-driven decision support](https://mlsanity.com/paper/2302.12389/) (2023) - [Do People Engage Cognitively with AI? Impact of AI Assistance on Incidental Learning](https://mlsanity.com/paper/2202.05402/) (2022) - [Knowing About Knowing: An Illusion of Human Competence Can Hinder Appropriate Reliance on AI Systems](https://mlsanity.com/paper/2301.11333/) (2023) - [Improving Human-AI Collaboration With Descriptions of AI Behavior](https://mlsanity.com/paper/2301.06937/) (2023) - [Selective Explanations: Leveraging Human Input to Align Explainable AI](https://mlsanity.com/paper/2301.09656/) (2023) - [Ground(less) Truth: A Causal Framework for Proxy Labels in Human-Algorithm Decision-Making](https://mlsanity.com/paper/2302.06503/) (2023) - [Effective Human-AI Teams via Learned Natural Language Rules and Onboarding](https://mlsanity.com/paper/2311.01007/) (2023) - [Learning When to Advise Human Decision Makers](https://mlsanity.com/paper/2209.13578/) (2022) - [Adapting User Interfaces with Model-based Reinforcement Learning](https://mlsanity.com/paper/2103.06807/) (2021) - [Accuracy-Time Tradeoffs in AI-Assisted Decision Making under Time Pressure](https://mlsanity.com/paper/2306.07458/) (2023) - [Sensing Wellbeing in the Workplace, Why and For Whom? Envisioning Impacts with Organizational Stakeholders](https://mlsanity.com/paper/2303.06794/) (2023) - [Overcoming Algorithm Aversion: A Comparison between Process and Outcome Control](https://mlsanity.com/paper/2303.12896/) (2023) - [Training Towards Critical Use: Learning to Situate AI Predictions Relative to Human Knowledge](https://mlsanity.com/paper/2308.15700/) (2023) ## Built on by (most cited first, 13 of 13) - [Survey on Large Language Model-Enhanced Reinforcement Learning: Concept, Taxonomy, and Methods](https://mlsanity.com/paper/2404.00282/) (2024) - [Modulating Language Model Experiences through Frictions](https://mlsanity.com/paper/2407.12804/) (2024) - [Utilizing Human Behavior Modeling to Manipulate Explanations in AI-Assisted Decision Making: The Good, the Bad, and the Scary](https://mlsanity.com/paper/2411.10461/) (2024) - [AI Assistance Reduces Persistence and Hurts Independent Performance](https://mlsanity.com/paper/2604.04721/) (2026) - [AI-Assisted Decision Making with Human Learning](https://mlsanity.com/paper/2502.13062/) (2025) - [Counterfactually Fair Reinforcement Learning via Sequential Data Preprocessing](https://mlsanity.com/paper/2501.06366/) (2025) - [BCR-DRL: Behavior- and Context-aware Reward for Deep Reinforcement Learning in Human-AI Coordination](https://mlsanity.com/paper/2408.07877/) (2024) - [Collaborative Causal Sensemaking: Closing the Complementarity Gap in Human-AI Decision Support](https://mlsanity.com/paper/2512.07801/) (2025) - [Dynamic Information Sub-Selection for Decision Support](https://mlsanity.com/paper/2410.23423/) (2024) - [Exploring the Impact of Explainable AI and Cognitive Capabilities on Users' Decisions](https://mlsanity.com/paper/2505.01192/) (2025) - [Achieving Trustworthy Real-Time Decision Support Systems with Low-Latency Interpretable AI Models](https://mlsanity.com/paper/2506.20018/) (2025) - [Learning to Decide with AI Assistance under Human-Alignment](https://mlsanity.com/paper/2605.12646/) (2026) - [Should I stay or should I show? Learning to selectively disclose information](https://mlsanity.com/paper/2609.39818/) (2026) ## Abstract (arXiv) Imagine if AI decision-support tools not only complemented our ability to make accurate decisions, but also improved our skills, boosted collaboration, and elevated the joy we derive from our tasks. Despite the potential to optimize a broad spectrum of such human-centric objectives, the design of current AI tools remains focused on decision accuracy alone. We propose offline reinforcement learning (RL) as a general approach for modeling human-AI decision-making to optimize human-AI interaction for diverse objectives. RL can optimize such objectives by tailoring decision support, providing the right type of assistance to the right person at the right time. We instantiated our approach with two objectives: human-AI accuracy on the decision-making task and human learning about the task and learned decision support policies from previous human-AI interaction data. We compared the optimized policies against several baselines in AI-assisted decision-making. Across two experiments (N=316 and N=964), our results demonstrated that people interacting with policies optimized for accuracy achieve significantly better accuracy -- and even human-AI complementarity -- compared to those interacting with any other type of AI support. Our results further indicated that human learning was more difficult to optimize than accuracy, with participants who interacted with learning-optimized policies showing significant learning improvement only at times. Our research (1) demonstrates offline RL to be a promising approach to model human-AI decision-making, leading to policies that may optimize human-centric objectives and provide novel insights about the AI-assisted decision-making space, and (2) emphasizes the importance of considering human-centric objectives beyond decision accuracy in AI-assisted decision-making, opening up the novel research challenge of optimizing human-AI interaction for such objectives.