# BCR-DRL: Behavior- and Context-aware Reward for Deep Reinforcement Learning in Human-AI Coordination

arXiv: https://arxiv.org/abs/2408.07877 · HTML: https://mlsanity.com/paper/2408.07877/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods.
- Builds on 19 of its 26 references that are in the map.
- First posted on arXiv 15 Aug 2024.

## Placed in

- Deep reinforcement learning methods / [Deep RL algorithms and representations](https://mlsanity.com/deeprl/deep-rl-algorithms-and-representations/)

## Builds on (most cited first, 19 of 19)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Asynchronous Methods for Deep Reinforcement Learning](https://mlsanity.com/paper/1602.01783/) (2016)
- [Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments](https://mlsanity.com/paper/1706.02275/) (2017)
- [Curiosity-driven Exploration by Self-supervised Prediction](https://mlsanity.com/paper/1705.05363/) (2017)
- [End-to-End Multi-Task Learning with Attention](https://mlsanity.com/paper/1803.10704/) (2018)
- [Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2003.08839/) (2020)
- [On the Utility of Learning about Humans for Human-AI Coordination](https://mlsanity.com/paper/1910.05789/) (2019)
- [Cooperative Inverse Reinforcement Learning](https://mlsanity.com/paper/1606.03137/) (2016)
- [Learning to Utilize Shaping Rewards: A New Approach of Reward Shaping](https://mlsanity.com/paper/2011.02669/) (2020)
- [Maximum Entropy Population-Based Training for Zero-Shot Human-AI Coordination](https://mlsanity.com/paper/2112.11701/) (2021)
- [The Effect of Modeling Human Rationality Level on Learning Rewards from Multiple Feedback Types](https://mlsanity.com/paper/2208.10687/) (2022)
- [Towards Optimizing Human-Centric Objectives in AI-Assisted Decision-Making With Offline Reinforcement Learning](https://mlsanity.com/paper/2403.05911/) (2024)
- [PantheonRL: A MARL Library for Dynamic Training Interactions](https://mlsanity.com/paper/2112.07013/) (2021)
- [Efficiently Combining Human Demonstrations and Interventions for Safe Training of Autonomous Systems in Real-Time](https://mlsanity.com/paper/1810.11545/) (2018)
- [A Hierarchical Approach to Population Training for Human-AI Collaboration](https://mlsanity.com/paper/2305.16708/) (2023)
- [Secure Deep Reinforcement Learning for Dynamic Resource Allocation in Wireless MEC Networks](https://mlsanity.com/paper/2312.08016/) (2023)
- [Hybrid-Task Meta-Learning: A GNN Approach for Scalable and Transferable Bandwidth Allocation](https://mlsanity.com/paper/2401.10253/) (2023)
- [A Constrained Deep Reinforcement Learning Optimization for Reliable Network Slicing in a Blockchain-Secured Low-Latency Wireless Network](https://mlsanity.com/paper/2403.09693/) (2024)

## Built on by (most cited first, 3 of 3)

- [Adaptive XAI in High Stakes Environments: Modeling Swift Trust with Multimodal Feedback in Human AI Teams](https://mlsanity.com/paper/2507.21158/) (2025)
- [Adaptive Human-AI Coordination via Hierarchical Action Disentanglement](https://mlsanity.com/paper/2605.24343/) (2026)
- [Partner-Aware Hierarchical Skill Discovery for Robust Human-AI Collaboration](https://mlsanity.com/paper/2605.24352/) (2026)

## Abstract (arXiv)

Deep reinforcement Learning (DRL) offers a powerful framework for training AI agents to coordinate with human partners. However, DRL faces two critical challenges in human-AI coordination (HAIC): sparse rewards and unpredictable human behaviors. These challenges significantly limit DRL to identify effective coordination policies, due to its impaired capability of optimizing exploration and exploitation. To address these limitations, we propose an innovative behavior- and context-aware reward (BCR) for DRL, which optimizes exploration and exploitation by leveraging human behaviors and contextual information in HAIC. Our BCR consists of two components: (i) A novel dual intrinsic rewarding scheme to enhance exploration. This scheme composes an AI self-motivated intrinsic reward and a human-motivated intrinsic reward, which are designed to increase the capture of sparse rewards by a logarithmic-based strategy; and (ii) A new context-aware weighting mechanism for the designed rewards to improve exploitation. This mechanism helps the AI agent prioritize actions that better coordinate with the human partner by utilizing contextual information that can reflect the evolution of learning. Extensive simulations in the Overcooked environment demonstrate that our approach can increase the cumulative sparse rewards by approximately 20%, and improve the sample efficiency by around 38% compared to state-of-the-art baselines.