mlsanityData to 7 Oct 2026

Learning to Make Adherence-Aware Advice

arXiv 2310.00817 ↗First posted 1 Oct 2023stat.ML cs.LGcited by 13

Not on the map. Why?

Abstract (arXiv)

As artificial intelligence (AI) systems play an increasingly prominent role in human decision-making, challenges surface in the realm of human-AI interactions. One challenge arises from the suboptimal AI policies due to the inadequate consideration of humans disregarding AI recommendations, as well as the need for AI to provide advice selectively when it is most pertinent. This paper presents a sequential decision-making model that (i) takes into account the human's adherence level (the probability that the human follows/rejects machine advice) and (ii) incorporates a defer option so that the machine can temporarily refrain from making advice. We provide learning algorithms that learn the optimal advice policy and make advice only at critical time stamps. Compared to problem-agnostic reinforcement learning algorithms, our specialized learning algorithms not only enjoy better theoretical convergence properties but also show strong empirical performance.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Builds on 24 of its 46 references that are in the map. First posted on arXiv 1 Oct 2023.

Builds on

24 of 46 references.

References · 24

  1. 2017Minimax Regret Bounds for Reinforcement Learningcited by 500
  2. 2018Is Q-learning Provably Efficient?cited by 485
  3. 2019On the Utility of Learning about Humans for Human-AI Coordinationcited by 345
  4. 2020Consistent Estimators for Learning to Defer to an Expertcited by 262
  5. 2019Tighter Problem-Dependent Regret Bounds in Reinforcement Learning without Domain Knowledge using Value Function Boundscited by 194
  6. 2020Reward-Free Exploration for Reinforcement Learningcited by 177
  7. 2017Unifying PAC and Regret: Uniform PAC Bounds for Episodic Reinforcement Learningcited by 175
  8. 2015Sample Complexity of Episodic Fixed-Horizon Reinforcement Learningcited by 105
  9. 2020Episodic Reinforcement Learning in Finite MDPs: Minimax Lower Bounds Revisitedcited by 104
  10. 2020Fast active learning for pure exploration in reinforcement learningcited by 92
  11. 2020Adaptive Reward-Free Explorationcited by 77
  12. 2021Differentiable Learning Under Triagecited by 74
12 more
  1. 2020Classification Under Human Assistancecited by 42
  2. 2020Task-agnostic Exploration in Reinforcement Learningcited by 37
  3. 2020A Sample-Efficient Algorithm for Episodic Finite-Horizon MDP with Constraintscited by 34
  4. 2023Learning to Reject with a Fixed Predictor: Application to Decontextualizationcited by 32
  5. 2022The Boltzmann Policy Distribution: Accounting for Systematic Suboptimality in Human Modelscited by 31
  6. 2021A Simple Reward-free Approach to Constrained Reinforcement Learningcited by 28
  7. 2020LESS is More: Rethinking Probabilistic Models of Human Behaviorcited by 17
  8. 2022The Best Decisions Are Not the Best Advice: Making Adherence-Aware Recommendationscited by 16
  9. 2021Reinforcement Learning Under Algorithmic Triagecited by 15
  10. 2020Modeling Trust in Human-Robot Interaction: A Surveycited by 12
  11. 2022Lazy-MDPs: Towards Interpretable Reinforcement Learning by Learning When to Actcited by 7
  12. 2022Algorithmic Decision-Making Safeguarded by Human Knowledgecited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 6; 2025: 4; 2026: 1.

13 papers.

Sort:

  1. 2024Learning to Route LLMs with Confidence Tokenscited by 66
  2. 2023Predictor-Rejector Multi-Class Abstention: Theoretical Analysis and Algorithmscited by 36
  3. 2023Theoretically Grounded Loss Functions and Algorithms for Score-Based Multi-Class Abstentioncited by 32
  4. 2024Realizable H-Consistent and Bayes-Consistent Loss Functions for Learning to Defercited by 26
  5. 2025Mastering Multiple-Expert Routing: Realizable H-Consistency and Strong Guarantees for Learning to Defercited by 16
  6. 2025Budgeted Multiple-Expert Deferralcited by 11
  7. 2024A Framework for Effective AI Recommendations in Cyber-Physical-Human Systemscited by 4
  8. 2024Optimizing Delegation in Collaborative Human-AI Hybrid Teamscited by 3
  9. 2025AI Recommendation Systems for Lane-Changing Using Adherence-Aware Reinforcement Learningcited by 3
  10. 2025Conformalized Decision Risk Assessmentcited by 3
  11. 2024Balancing Optimality and Diversity: Human-Centered Decision Making through Generative Curationcited by 2
  12. 2024Optimizing Risk-averse Human-AI Hybrid Teamscited by 1
1 more
  1. 2026Improving Human Performance with Value-Aware Interventions: A Case Study in Chesscited by 0