# Convergence of Learning Dynamics in Stackelberg Games

arXiv: https://arxiv.org/abs/1906.01217 · HTML: https://mlsanity.com/paper/1906.01217/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Jun 2019.

## Built on by (most cited first, 28 of 28)

- [On the Principles of Parsimony and Self-Consistency for the Emergence of Intelligence](https://mlsanity.com/paper/2207.04630/) (2022)
- [Sample-Efficient Learning of Stackelberg Equilibria in General-Sum Games](https://mlsanity.com/paper/2102.11494/) (2021)
- [PARL: A Unified Framework for Policy Alignment in Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2308.02585/) (2023)
- [Calibrated Stackelberg Games: Learning Optimal Commitments Against Calibrated Agents](https://mlsanity.com/paper/2306.02704/) (2023)
- [Can Reinforcement Learning Find Stackelberg-Nash Equilibria in General-Sum Markov Games with Myopic Followers?](https://mlsanity.com/paper/2112.13521/) (2021)
- [ACC-Collab: An Actor-Critic Approach to Multi-Agent LLM Collaboration](https://mlsanity.com/paper/2411.00053/) (2024)
- [A Ranking Game for Imitation Learning](https://mlsanity.com/paper/2202.03481/) (2022)
- [Multi-Rate VAE: Train Once, Get the Full Rate-Distortion Curve](https://mlsanity.com/paper/2212.03905/) (2022)
- [A review of Generative Adversarial Networks for Electronic Health Records: applications, evaluation measures and data sources](https://mlsanity.com/paper/2203.07018/) (2022)
- [Two-sided Competing Matching Recommendation Markets With Quota and Complementary Preferences Constraints](https://mlsanity.com/paper/2301.10230/) (2023)
- [RotoGBML: Towards Out-of-Distribution Generalization for Gradient-Based Meta-Learning](https://mlsanity.com/paper/2303.06679/) (2023)
- [Impact of Decentralized Learning on Player Utilities in Stackelberg Games](https://mlsanity.com/paper/2403.00188/) (2024)
- [Is Knowledge Power? On the (Im)possibility of Learning from Strategic Interactions](https://mlsanity.com/paper/2408.08272/) (2024)
- [Pursuit of a Discriminative Representation for Multiple Subspaces via Sequential Games](https://mlsanity.com/paper/2206.09120/) (2022)
- [On Characterizing GAN Convergence Through Proximal Duality Gap](https://mlsanity.com/paper/2105.04801/) (2021)
- [A Survey of Decision Making in Adversarial Games](https://mlsanity.com/paper/2207.07971/) (2022)
- [Stackelberg Driver Model for Continual Policy Improvement in Scenario-Based Closed-Loop Autonomous Driving](https://mlsanity.com/paper/2309.14235/) (2023)
- [Robust Forecasting for Robotic Control: A Game-Theoretic Approach](https://mlsanity.com/paper/2209.10802/) (2022)
- [Recursive Reasoning in Minimax Games: A Level k Gradient Play Method](https://mlsanity.com/paper/2210.16482/) (2022)
- [PAPAL: A Provable PArticle-based Primal-Dual ALgorithm for Mixed Nash Equilibrium](https://mlsanity.com/paper/2303.00970/) (2023)
- [ApproxED: Approximate exploitability descent via learned best responses](https://mlsanity.com/paper/2301.08830/) (2023)
- [MESOB: Balancing Equilibria & Social Optimality](https://mlsanity.com/paper/2307.07911/) (2023)
- [ADDS: Adaptive Differentiable Sampling for Robust Multi-Party Learning](https://mlsanity.com/paper/2110.15522/) (2021)
- [Follower Agnostic Methods for Stackelberg Games](https://mlsanity.com/paper/2302.01421/) (2023)
- [Uncoupled Learning of Differential Stackelberg Equilibria with Commitments](https://mlsanity.com/paper/2302.03438/) (2023)
- [Simulate and Optimise: A two-layer mortgage simulator for designing novel mortgage assistance products](https://mlsanity.com/paper/2411.00563/) (2024)
- [HERAKLES: Hierarchical Skill Compilation for Open-ended LLM Agents](https://mlsanity.com/paper/2508.14751/) (2025)
- [Finite-Time Analysis of Q-Value Iteration for General-Sum Stackelberg Games](https://mlsanity.com/paper/2604.04394/) (2026)

## Abstract (arXiv)

This paper investigates the convergence of learning dynamics in Stackelberg games. In the class of games we consider, there is a hierarchical game being played between a leader and a follower with continuous action spaces. We establish a number of connections between the Nash and Stackelberg equilibrium concepts and characterize conditions under which attracting critical points of simultaneous gradient descent are Stackelberg equilibria in zero-sum games. Moreover, we show that the only stable critical points of the Stackelberg gradient dynamics are Stackelberg equilibria in zero-sum games. Using this insight, we develop a gradient-based update for the leader while the follower employs a best response strategy for which each stable critical point is guaranteed to be a Stackelberg equilibrium in zero-sum games. As a result, the learning rule provably converges to a Stackelberg equilibria given an initialization in the region of attraction of a stable critical point. We then consider a follower employing a gradient-play update rule instead of a best response strategy and propose a two-timescale algorithm with similar asymptotic convergence guarantees. For this algorithm, we also provide finite-time high probability bounds for local convergence to a neighborhood of a stable Stackelberg equilibrium in general-sum games. Finally, we present extensive numerical results that validate our theory, provide insights into the optimization landscape of generative adversarial networks, and demonstrate that the learning dynamics we propose can effectively train generative adversarial networks.