Risk Sensitivity in Markov Games and Multi-Agent Reinforcement Learning: A Systematic Review
Not on the map. Why?
Abstract (arXiv)
Markov games (MGs) and multi-agent reinforcement learning (MARL) are studied to model decision making in multi-agent systems. Traditionally, the objective in MG and MARL has been risk-neutral, i.e., agents are assumed to optimize a performance metric such as expected return, without taking into account subjective or cognitive preferences of themselves or of other agents. However, ignoring such preferences leads to inaccurate models of decision making in many real-world scenarios in finance, operations research, and behavioral economics. Therefore, when these preferences are present, it is necessary to incorporate a suitable measure of risk into the optimization objective of agents, which opens the door to risk-sensitive MG and MARL. In this paper, we systemically review the literature on risk sensitivity in MG and MARL that has been growing in recent years alongside other areas of reinforcement learning and game theory. We define and mathematically describe different risk measures used in MG and MARL and individually for each measure, discuss articles that incorporate it. Finally, we identify recent trends in theoretical and applied works in the field and discuss possible directions of future research.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 40 of its 113 references that are in the map. First posted on arXiv 10 Jun 2024.
Builds on
40 of 113 references.
References · 40
- 2015Continuous control with deep reinforcement learningcited by 3,407
- 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
- 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,258
- 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,558
- 2017A Distributional Perspective on Reinforcement Learningcited by 765
- 2019Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithmscited by 567
- 2018Implicit Quantile Networks for Distributional Reinforcement Learningcited by 369
- 2018Deep Reinforcement Learning for Multi-Agent Systems: A Review of Challenges, Solutions and Applicationscited by 127
- 2020Game-Theoretic Multiagent Reinforcement Learningcited by 94
- 2021Global Convergence of Multi-Agent Policy Gradient in Markov Potential Gamescited by 93
- 2020Model-Based Multi-Agent RL in Zero-Sum Markov Games with Near-Optimal Sample Complexitycited by 92
- 2021Decentralized Q-Learning in Zero-sum Markov Gamescited by 58
28 more
- 2022Independent Policy Gradient for Large-Scale Markov Potential Games: Sharper Rates, Function Approximation, and Game-Agnostic Convergencecited by 57
- 2021Independent Natural Policy Gradient Always Converges in Markov Potential Gamescited by 37
- 2021RMIX: Learning Risk-Sensitive Policies for Cooperative Reinforcement Learning Agentscited by 31
- 2021Learning in Nonzero-Sum Stochastic Games with Potentialscited by 26
- 2019Stochastic Dynamic Games in Belief Spacecited by 23
- 2022Independent and Decentralized Learning in Markov Potential Gamescited by 22
- 2023RiskQ: Risk-sensitive Multi-Agent Reinforcement Learning Value Factorizationcited by 20
- 2023A Finite-Sample Analysis of Payoff-Based Independent Learning in Zero-Sum Stochastic Gamescited by 15
- 2020Distributionally Robust Markov Decision Processes and their Connection to Risk Measurescited by 14
- 2013Predicting Human Behavior in Unrepeated, Simultaneous-Move Gamescited by 10
- 2023Multi-Player Zero-Sum Markov Games with Networked Separable Interactionscited by 10
- 2022Provably Efficient Fictitious Play Policy Optimization for Zero-Sum Markov Games with Structured Transitionscited by 9
- 2015On the Dual Representation of Coherent Risk Measurescited by 8
- 2021Policy Gradient Methods for Distortion Risk Measurescited by 6
- 2019Leader-Follower Network Aggregative Game with Stochastic Agents' Communication and Activenesscited by 5
- 2020Risk-Aware Energy Scheduling for Edge Computing with Microgrid: A Multi-Agent Deep Reinforcement Learning Approachcited by 5
- 2022Risk-Averse No-Regret Learning in Online Convex Gamescited by 5
- 2021Ultra-Reliable Indoor Millimeter Wave Communications using Multiple Artificial Intelligence-Powered Intelligent Surfacescited by 2
- 2021Nonzero-sum risk-sensitive continuous-time stochastic games with ergodic costscited by 2
- 2022Discrete-time Zero-Sum Games for Markov chains with risk-sensitive average cost criterioncited by 2
- 2016Nonzero-Sum Risk Sensitive Stochastic Games for Continuous Time Markov Chainscited by 1
- 2016Nonzero-sum Risk-sensitive Stochastic Games on a Countable State Spacecited by 1
- 2020Discrete-time mean field games with risk averse-agentscited by 1
- 2021Zero and Non-zero Sum Risk-sensitive Semi-Markov Gamescited by 1
- 2022Chance-Constrained Iterative Linear-Quadratic Stochastic Gamescited by 1
- 2023Entropic Risk for Turn-Based Stochastic Gamescited by 1
- 2023Generalized Stochastic Dynamic Aggregative Game for Demand-Side Management in Microgrids with Shared Batterycited by 1
- 2024Risk-Sensitive Multi-Agent Reinforcement Learning in Network Aggregative Markov Gamescited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025A Survey of Safe Reinforcement Learning and Constrained MDPs: A Technical Survey on Single-Agent and Multi-Agent Safety
- 2024Pessimism Meets Risk: Risk-Sensitive Offline Reinforcement Learning
- 2024Taming Equilibrium Bias in Risk-Sensitive Multi-Agent Reinforcement Learning
- 2025Markov Potential Game Construction and Multi-Agent Reinforcement Learning with Applications to Autonomous Driving ↗
- 2024Tractable Equilibrium Computation in Markov Games through Risk Aversion
- 2026Decoupling Time and Risk: Risk-Sensitive Reinforcement Learning with General Discounting
- 2025Multi-Agent Reinforcement Learning for Resources Allocation Optimization: A Survey
- 2026Provably Convergent Actor-Critic for MARL through Risk-aversion