mlsanityData to 7 Oct 2026

Taming Equilibrium Bias in Risk-Sensitive Multi-Agent Reinforcement Learning

arXiv 2405.02724 ↗First posted 4 May 2024cs.LG cs.GTcited by 0

Not on the map. Why?

Abstract (arXiv)

We study risk-sensitive multi-agent reinforcement learning under general-sum Markov games, where agents optimize the entropic risk measure of rewards with possibly diverse risk preferences. We show that using the regret naively adapted from existing literature as a performance metric could induce policies with equilibrium bias that favor the most risk-sensitive agents and overlook the other agents. To address such deficiency of the naive regret, we propose a novel notion of regret, which we call risk-balanced regret, and show through a lower bound that it overcomes the issue of equilibrium bias. Furthermore, we develop a self-play algorithm for learning Nash, correlated, and coarse correlated equilibria in risk-sensitive Markov games. We prove that the proposed algorithm attains near-optimal regret guarantees with respect to the risk-balanced regret.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 15 of its 30 references that are in the map. First posted on arXiv 4 May 2024.

Builds on

15 of 30 references.

References · 15

  1. 2020Provable Self-Play Algorithms for Competitive Reinforcement Learningcited by 112
  2. 2020Near-Optimal Reinforcement Learning with Self-Playcited by 108
  3. 2020A Sharp Analysis of Model-based Reinforcement Learning with Self-Playcited by 104
  4. 2020Learning Zero-Sum Simultaneous-Move Markov Games Using Function Approximation and Correlated Equilibriumcited by 100
  5. 2020Model-Based Multi-Agent RL in Zero-Sum Markov Games with Near-Optimal Sample Complexitycited by 91
  6. 2020Risk-Sensitive Reinforcement Learning: Near-Optimal Risk-Sample Tradeoff in Regretcited by 57
  7. 2021Exponential Bellman Equation and Improved Regret Bounds for Risk-Sensitive Reinforcement Learningcited by 42
  8. 2022Learn to Match with No Regret: Reinforcement Learning in Markov Matching Marketscited by 16
  9. 2021Convergence and Alignment of Gradient Descent with Random Backpropagation Weightscited by 14
  10. 2018On the Landscape of Synchronization Networks: A Perspective from Nonconvex Optimizationcited by 11
  11. 2020Meta Learning in the Continuous Time Limitcited by 10
  12. 2020Online Learning in Unknown Markov Gamescited by 9
3 more
  1. 2023Finding Regularized Competitive Equilibria of Heterogeneous Agent Macroeconomic Models with Reinforcement Learningcited by 4
  2. 2016Zero-sum Risk-Sensitive Stochastic Gamescited by 2
  3. 2022Cascaded Gaps: Towards Gap-Dependent Regret for Risk-Sensitive Reinforcement Learningcited by 2

Built on by

Not cited yet.