# Optimizing Risk-averse Human-AI Hybrid Teams

arXiv: https://arxiv.org/abs/2403.08386 · HTML: https://mlsanity.com/paper/2403.08386/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 35 references that are in the map.
- First posted on arXiv 13 Mar 2024.

## Builds on (most cited first, 16 of 16)

- [On the Utility of Learning about Humans for Human-AI Coordination](https://mlsanity.com/paper/1910.05789/) (2019)
- [Discovering and Validating AI Errors With Crowdsourced Failure Reports](https://mlsanity.com/paper/2109.11690/) (2021)
- [Deep Reinforcement Learning for Event-Driven Multi-Agent Decision Processes](https://mlsanity.com/paper/1709.06656/) (2017)
- [Option-Critic in Cooperative Multi-agent Systems](https://mlsanity.com/paper/1911.12825/) (2019)
- [Collective Intelligence in Human-AI Teams: A Bayesian Theory of Mind Approach](https://mlsanity.com/paper/2208.11660/) (2022)
- [Reinforcement Learning Under Algorithmic Triage](https://mlsanity.com/paper/2109.11328/) (2021)
- [LDSA: Learning Dynamic Subtask Assignment in Cooperative Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2205.02561/) (2022)
- [Learning to Make Adherence-Aware Advice](https://mlsanity.com/paper/2310.00817/) (2023)
- [Modeling Trust in Human-Robot Interaction: A Survey](https://mlsanity.com/paper/2011.04796/) (2020)
- [Decentralized Cooperative Planning for Automated Vehicles with Hierarchical Monte Carlo Tree Search](https://mlsanity.com/paper/1807.09530/) (2018)
- [Optimizing delegation between human and AI collaborative agents](https://mlsanity.com/paper/2309.14718/) (2023)
- [A Cognitive Framework for Delegation Between Error-Prone AI and Human Agents](https://mlsanity.com/paper/2204.02889/) (2022)
- [Developing cooperative policies for multi-stage reinforcement learning tasks](https://mlsanity.com/paper/2205.05230/) (2022)
- [Compensating for Sensing Failures via Delegation in Human-AI Hybrid Systems](https://mlsanity.com/paper/2303.01300/) (2023)
- [Learning Reusable Options for Multi-Task Reinforcement Learning](https://mlsanity.com/paper/2001.01577/) (2020)
- [Optimizing Delegation in Collaborative Human-AI Hybrid Teams](https://mlsanity.com/paper/2402.05605/) (2024)

## Built on by (most cited first, 1 of 1)

- [PLACO: A Multi-Stage Framework for Cost-Effective Performance in Human-AI Teams](https://mlsanity.com/paper/2605.08388/) (2026)

## Abstract (arXiv)

We anticipate increased instances of humans and AI systems working together in what we refer to as a hybrid team. The increase in collaboration is expected as AI systems gain proficiency and their adoption becomes more widespread. However, their behavior is not error-free, making hybrid teams a very suitable solution. As such, we consider methods for improving performance for these teams of humans and AI systems. For hybrid teams, we will refer to both the humans and AI systems as agents. To improve team performance over that seen for agents operating individually, we propose a manager which learns, through a standard Reinforcement Learning scheme, how to best delegate, over time, the responsibility of taking a decision to any of the agents. We further guide the manager's learning so they also minimize how many changes in delegation are made resulting from undesirable team behavior. We demonstrate the optimality of our manager's performance in several grid environments which include failure states which terminate an episode and should be avoided. We perform our experiments with teams of agents with varying degrees of acceptable risk, in the form of proximity to a failure state, and measure the manager's ability to make effective delegation decisions with respect to its own risk-based constraints, then compare these to the optimal decisions. Our results show our manager can successfully learn desirable delegations which result in team paths near/exactly optimal with respect to path length and number of delegations.