# Risk-Sensitive Bayesian Games for Multi-Agent Reinforcement Learning under Policy Uncertainty

arXiv: https://arxiv.org/abs/2203.10045 · HTML: https://mlsanity.com/paper/2203.10045/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 32 references that are in the map.
- First posted on arXiv 18 Mar 2022.

## Builds on (most cited first, 12 of 12)

- [Multi-Task Learning as Multi-Objective Optimization](https://mlsanity.com/paper/1810.04650/) (2018)
- [A Unified Game-Theoretic Approach to Multiagent Reinforcement Learning](https://mlsanity.com/paper/1711.00832/) (2017)
- [Deep Reinforcement Learning from Self-Play in Imperfect-Information Games](https://mlsanity.com/paper/1603.01121/) (2016)
- [Risk-sensitive Reinforcement Learning](https://mlsanity.com/paper/1311.2097/) (2013)
- [Bayesian Exploration: Incentivizing Exploration in Bayesian Games](https://mlsanity.com/paper/1602.07570/) (2016)
- [Risk-Averse Bayes-Adaptive Reinforcement Learning](https://mlsanity.com/paper/2102.05762/) (2021)
- [Epistemic Risk-Sensitive Reinforcement Learning](https://mlsanity.com/paper/1906.06273/) (2019)
- [SENTINEL: Taming Uncertainty with Ensemble-based Distributional Reinforcement Learning](https://mlsanity.com/paper/2102.11075/) (2021)
- [Dynamic Bayesian Games for Adversarial and Defensive Cyber Deception](https://mlsanity.com/paper/1809.02013/) (2018)
- [Bayesian Persuasive Driving](https://mlsanity.com/paper/1809.09735/) (2018)
- [Convergence Analysis of Iterated Best Response for a Trusted Computation Game](https://mlsanity.com/paper/1611.02329/) (2016)
- [Risk-Sensitive Cooperative Games for Human-Machine Systems](https://mlsanity.com/paper/1705.09580/) (2017)

## Built on by (most cited first, 5 of 5)

- [A Game-Theoretic Framework for Managing Risk in Multi-Agent Systems](https://mlsanity.com/paper/2205.15434/) (2022)
- [Performative Policy Gradient: Optimality in Performative Reinforcement Learning](https://mlsanity.com/paper/2512.20576/) (2025)
- [Tractable Equilibrium Computation in Markov Games through Risk Aversion](https://mlsanity.com/paper/2406.14156/) (2024)
- [Training Generalizable Collaborative Agents via Strategic Risk Aversion](https://mlsanity.com/paper/2602.21515/) (2026)
- [The Fair Game: Auditing & Debiasing AI Algorithms Over Time](https://mlsanity.com/paper/2508.06443/) (2025)

## Abstract (arXiv)

In stochastic games with incomplete information, the uncertainty is evoked by the lack of knowledge about a player's own and the other players' types, i.e. the utility function and the policy space, and also the inherent stochasticity of different players' interactions. In existing literature, the risk in stochastic games has been studied in terms of the inherent uncertainty evoked by the variability of transitions and actions. In this work, we instead focus on the risk associated with the textituncertainty over types. We contrast this with the multi-agent reinforcement learning framework where the other agents have fixed stationary policies and investigate risk-sensitiveness due to the uncertainty about the other agents' adaptive policies. We propose risk-sensitive versions of existing algorithms proposed for risk-neutral stochastic games, such as Iterated Best Response (IBR), Fictitious Play (FP) and a general multi-objective gradient approach using dual ascent (DAPG). Our experimental analysis shows that risk-sensitive DAPG performs better than competing algorithms for both social welfare and general-sum stochastic games.