# Confounding-Robust Policy Improvement with Human-AI Teams

arXiv: https://arxiv.org/abs/2310.08824 · HTML: https://mlsanity.com/paper/2310.08824/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 56 references that are in the map.
- First posted on arXiv 13 Oct 2023.

## Builds on (most cited first, 29 of 29)

- [Predict Responsibly: Improving Fairness and Accuracy by Learning to Defer](https://mlsanity.com/paper/1711.06664/) (2017)
- [Doubly Robust Policy Evaluation and Optimization](https://mlsanity.com/paper/1503.02834/) (2015)
- [Learning to Complement Humans](https://mlsanity.com/paper/2005.00582/) (2020)
- [The Algorithmic Automation Problem: Prediction, Triage, and Human Effort](https://mlsanity.com/paper/1903.12220/) (2019)
- [Confounding-Robust Policy Improvement](https://mlsanity.com/paper/1805.08593/) (2018)
- [Sensitivity Analysis of Individual Treatment Effects: A Robust Conformal Inference Approach](https://mlsanity.com/paper/2111.12161/) (2021)
- [Off-policy Policy Evaluation For Sequential Decisions Under Unobserved Confounding](https://mlsanity.com/paper/2003.05623/) (2020)
- [Balanced Policy Evaluation and Learning](https://mlsanity.com/paper/1705.07384/) (2017)
- [Regression Under Human Assistance](https://mlsanity.com/paper/1909.02963/) (2019)
- [Conformal Sensitivity Analysis for Individual Treatment Effects](https://mlsanity.com/paper/2112.03493/) (2021)
- [Bounds on the conditional and average treatment effect with unobserved confounding factors](https://mlsanity.com/paper/1808.09521/) (2018)
- [Human-AI Collaboration with Bandit Feedback](https://mlsanity.com/paper/2105.10614/) (2021)
- [Sensitivity analysis for inverse probability weighting estimators via the percentile bootstrap](https://mlsanity.com/paper/1711.11286/) (2017)
- [Sharp Sensitivity Analysis for Inverse Propensity Weighting via Quantile Balancing](https://mlsanity.com/paper/2102.04543/) (2021)
- [Sense and Sensitivity Analysis: Simple Post-Hoc Analysis of Bias Due to Unobserved Confounding](https://mlsanity.com/paper/2003.01747/) (2020)
- [More Efficient Policy Learning via Optimal Retargeting](https://mlsanity.com/paper/1906.08611/) (2019)
- [Policy Learning with Observational Data](https://mlsanity.com/paper/1702.02896/) (2017)
- [Toward Supporting Perceptual Complementarity in Human-AI Collaboration via Reflection on Unobservables](https://mlsanity.com/paper/2207.13834/) (2022)
- [Balanced off-policy evaluation in general action spaces](https://mlsanity.com/paper/1906.03694/) (2019)
- [Partial Identification with Noisy Covariates: A Robust Optimization Approach](https://mlsanity.com/paper/2202.10665/) (2022)
- [Policy Learning under Biased Sample Selection](https://mlsanity.com/paper/2304.11735/) (2023)
- [A Case for Backward Compatibility for Human-AI Teams](https://mlsanity.com/paper/1906.01148/) (2019)
- [Learning Complementary Policies for Human-AI Teams](https://mlsanity.com/paper/2302.02944/) (2023)
- [Augmented Fairness: An Interpretable Model Augmenting Decision-Makers' Fairness](https://mlsanity.com/paper/2011.08398/) (2020)
- [Loss Functions for Discrete Contextual Pricing with Observational Data](https://mlsanity.com/paper/2111.09933/) (2021)
- [Enhancing Counterfactual Classification via Self-Training](https://mlsanity.com/paper/2112.04461/) (2021)
- [Balanced Off-Policy Evaluation for Personalized Pricing](https://mlsanity.com/paper/2302.12736/) (2023)
- [Classifying Treatment Responders Under Causal Effect Monotonicity](https://mlsanity.com/paper/1902.05482/) (2019)
- [Minimax Optimal Estimation of Stability Under Distribution Shift](https://mlsanity.com/paper/2212.06338/) (2022)

## Built on by (most cited first, 4 of 4)

- [Predictive Performance Comparison of Decision Policies Under Confounding](https://mlsanity.com/paper/2404.00848/) (2024)
- [HR-Bandit: Human-AI Collaborated Linear Recourse Bandit](https://mlsanity.com/paper/2410.14640/) (2024)
- [LIBRA: Language Model Informed Bandit Recourse Algorithm for Personalized Treatment Planning](https://mlsanity.com/paper/2601.11905/) (2026)
- [Set-Valued Policy Learning](https://mlsanity.com/paper/2605.19830/) (2026)

## Abstract (arXiv)

Human-AI collaboration has the potential to transform various domains by leveraging the complementary strengths of human experts and Artificial Intelligence (AI) systems. However, unobserved confounding can undermine the effectiveness of this collaboration, leading to biased and unreliable outcomes. In this paper, we propose a novel solution to address unobserved confounding in human-AI collaboration by employing sensitivity analysis from causal inference. Our approach combines domain expertise with AI-driven statistical modeling to account for potentially hidden confounders. We present a deferral collaboration framework for incorporating the sensitivity model into offline policy learning, enabling the system to control for the influence of unobserved confounding factors. In addition, we propose a personalized deferral collaboration system to leverage the diverse expertise of different human decision-makers. By adjusting for potential biases, our proposed solution enhances the robustness and reliability of collaborative outcomes. The empirical and theoretical analyses demonstrate the efficacy of our approach in mitigating unobserved confounding and improving the overall performance of human-AI collaborations.