# On Passivity, Reinforcement Learning and Higher-Order Learning in Multi-Agent Finite Games

arXiv: https://arxiv.org/abs/1808.04464 · HTML: https://mlsanity.com/paper/1808.04464/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Aug 2018.

## Built on by (most cited first, 8 of 8)

- [The Confluence of Networks, Games and Learning](https://mlsanity.com/paper/2105.08158/) (2021)
- [A Survey on Physics Informed Reinforcement Learning: Review and Open Problems](https://mlsanity.com/paper/2309.01909/) (2023)
- [Improved Soft Actor-Critic: Mixing Prioritized Off-Policy Samples with On-Policy Experience](https://mlsanity.com/paper/2109.11767/) (2021)
- [Boosting Exploration in Actor-Critic Algorithms by Incentivizing Plausible Novel States](https://mlsanity.com/paper/2210.00211/) (2022)
- [Accelerated regularized learning in finite N-person games](https://mlsanity.com/paper/2412.20365/) (2024)
- [Second-Order Mirror Descent: Convergence in Games Beyond Averaging and Discounting](https://mlsanity.com/paper/2111.09982/) (2021)
- [Integrating Lagrangian Neural Networks into the Dyna Framework for Reinforcement Learning](https://mlsanity.com/paper/2603.08468/) (2026)
- [Homomorphic Mappings for Value-Preserving State Aggregation in Markov Decision Processes](https://mlsanity.com/paper/2510.09965/) (2025)

## Abstract (arXiv)

In this paper, we propose a passivity-based methodology for analysis and design of reinforcement learning in multi-agent finite games. Starting from a known exponentially-discounted reinforcement learning scheme, we show that convergence to a Nash distribution can be shown in the class of games characterized by the monotonicity property of their (negative) payoff. We further exploit passivity to propose a class of higher-order schemes that preserve convergence properties, can improve the speed of convergence and can even converge in cases whereby their first-order counterpart fail to converge. We demonstrate these properties through numerical simulations for several representative games.