Online Decision Mediation
Not on the map. Why?
Abstract (arXiv)
Consider learning a decision support assistant to serve as an intermediary between (oracle) expert behavior and (imperfect) human behavior: At each time, the algorithm observes an action chosen by a fallible agent, and decides whether to *accept* that agent's decision, *intervene* with an alternative, or *request* the expert's opinion. For instance, in clinical diagnosis, fully-autonomous machine behavior is often beyond ethical affordances, thus real-world decision support is often limited to monitoring and forecasting. Instead, such an intermediary would strike a prudent balance between the former (purely prescriptive) and latter (purely descriptive) approaches, while providing an efficient interface between human mistakes and expert feedback. In this work, we first formalize the sequential problem of *online decision mediation* -- that is, of simultaneously learning and evaluating mediator policies from scratch with *abstentive feedback*: In each round, deferring to the oracle obviates the risk of error, but incurs an upfront penalty, and reveals the otherwise hidden expert action as a new training data point. Second, we motivate and propose a solution that seeks to trade off (immediate) loss terms against (future) improvements in generalization error; in doing so, we identify why conventional bandit algorithms may fail. Finally, through experiments and sensitivities on a variety of datasets, we illustrate consistent gains over applicable benchmarks on performance measures with respect to the mediator policy, the learned model, and the decision-making system as a whole.
Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 42 of its 105 references that are in the map. First posted on arXiv 28 Oct 2023.
Builds on
42 of 105 references.
References · 42
- 2015Trust Region Policy Optimizationcited by 2,503
- 2016OpenAI Gymcited by 1,684
- 2017Deep Bayesian Active Learning with Image Datacited by 812
- 2018GPyTorch: Blackbox Matrix-Matrix Gaussian Process Inference with GPU Accelerationcited by 603
- 2019BatchBALD: Efficient and Diverse Batch Acquisition for Deep Bayesian Active Learningcited by 428
- 2017A Tutorial on Thompson Samplingcited by 420
- 2016Cooperative Inverse Reinforcement Learningcited by 298
- 2020Consistent Estimators for Learning to Defer to an Expertcited by 262
- 2017Deep Reinforcement Learning framework for Autonomous Drivingcited by 224
- 2020Open Problems in Cooperative AIcited by 167
- 2021Machine Learning with a Reject Option: A surveycited by 107
- 2019Asking Easy Questions: A User-Friendly Approach to Active Reward Learningcited by 67
30 more
- 2020Classification with Rejection Based on Cost-sensitive Classificationcited by 55
- 2018Dirichlet-based Gaussian Processes for Large-scale Calibrated Classificationcited by 50
- 2021Interactive Label Cleaning with Example-based Explanationscited by 48
- 2019On the Calibration of Multiclass Classification with Rejectioncited by 41
- 2020Information Directed Sampling for Linear Partial Monitoringcited by 39
- 2015Upper-Confidence-Bound Algorithms for Active Learning in Multi-Armed Banditscited by 29
- 2021Teaching Humans When To Defer to a Classifier via Exemplarscited by 29
- 2017Online Learning with Abstentioncited by 28
- 2021Causal-BALD: Deep Bayesian Active Learning of Outcomes to Infer Treatment-Effects from Observational Datacited by 27
- 2021Exponential Savings in Agnostic Active Learning through Abstentioncited by 23
- 2015Active Learning from Weak and Strong Labelerscited by 22
- 2020Bandits with Partially Observable Confounded Datacited by 22
- 2020Active Preference Learning using Maximum Regretcited by 21
- 2022Efficient Active Learning with Abstentioncited by 21
- 2019Learning to Interactively Learn and Assistcited by 20
- 2020Stopping criterion for active learning based on deterministic generalization boundscited by 20
- 2020Fast Rates for Online Prediction with Abstentioncited by 19
- 2018An Efficient, Generalized Bellman Update For Cooperative Inverse Reinforcement Learningcited by 15
- 2016Active Learning from Imperfect Labelerscited by 13
- 2018Active Inverse Reward Designcited by 12
- 2020Learning in the Wild with Incremental Skeptical Gaussian Processescited by 12
- 2021Inverse Contextual Bandits: Learning How Behavior Evolves over Timecited by 12
- 2016The Extended Littlestone's Dimension for Learning with Mistakes and Abstentionscited by 10
- 2021Analysis of Thompson Sampling for Partially Observable Contextual Multi-Armed Banditscited by 10
- 2018Disease-Atlas: Navigating Disease Trajectories with Deep Learningcited by 8
- 2013Building Bridges: Viewing Active Learning from the Multi-Armed Bandit Lenscited by 6
- 2017A Contextual Bandit Approach for Stream-Based Active Learningcited by 2
- 2019Active Learning for Binary Classification with Abstentioncited by 2
- 2021Apple Tasting Revisited: Bayesian Approaches to Partially Monitored Online Binary Classificationcited by 2
- 2021Prescriptive Machine Learning for Automated Decision Making: Challenges and Opportunitiescited by 2
Built on by
Citing papers per year: 2023: 1; 2024: 0; 2025: 3.
4 papers.