# ReLExS: Reinforcement Learning Explanations for Stackelberg No-Regret Learners

arXiv: https://arxiv.org/abs/2408.14086 · HTML: https://mlsanity.com/paper/2408.14086/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL for games and combinatorial optimization, one of 15 directions in Deep reinforcement learning methods.
- Builds on 8 of its 25 references that are in the map.
- First posted on arXiv 26 Aug 2024.

## Placed in

- Deep reinforcement learning methods / [RL for games and combinatorial optimization](https://mlsanity.com/deeprl/rl-for-games-and-combinatorial-optimization/)

## Builds on (most cited first, 8 of 8)

- [Strategizing against No-regret Learners](https://mlsanity.com/paper/1909.13861/) (2019)
- [Robust Reinforcement Learning as a Stackelberg Game via Adaptively-Regularized Adversarial Training](https://mlsanity.com/paper/2202.09514/) (2022)
- [Can Reinforcement Learning Find Stackelberg-Nash Equilibria in General-Sum Markov Games with Myopic Followers?](https://mlsanity.com/paper/2112.13521/) (2021)
- [M3RL: Mind-aware Multi-agent Management Reinforcement Learning](https://mlsanity.com/paper/1810.00147/) (2018)
- [Adaptive Incentive Design with Multi-Agent Meta-Gradient Reinforcement Learning](https://mlsanity.com/paper/2112.10859/) (2021)
- [Oracles & Followers: Stackelberg Equilibria in Deep Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2210.11942/) (2022)
- [Solving Structured Hierarchical Games Using Differential Backward Induction](https://mlsanity.com/paper/2106.04663/) (2021)
- [The computation of approximate feedback Stackelberg equilibria in multi-player nonlinear constrained dynamic games](https://mlsanity.com/paper/2401.15745/) (2024)

## Abstract (arXiv)

With the constraint of a no regret follower, will the players in a two-player Stackelberg game still reach Stackelberg equilibrium? We first show when the follower strategy is either reward-average or transform-reward-average, the two players can always get the Stackelberg Equilibrium. Then, we extend that the players can achieve the Stackelberg equilibrium in the two-player game under the no regret constraint. Also, we show a strict upper bound of the follower's utility difference between with and without no regret constraint. Moreover, in constant-sum two-player Stackelberg games with non-regret action sequences, we ensure the total optimal utility of the game remains also bounded.