# The Lagrangian Method for Solving Constrained Markov Games

arXiv: https://arxiv.org/abs/2503.10561 · HTML: https://mlsanity.com/paper/2503.10561/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Mar 2025.

## Built on by (most cited first, 1 of 1)

- [Learning to Harvest Without Collapse in a Regenerative Commons: A Lagrangian Framework](https://mlsanity.com/paper/2609.36478/) (2026)

## Abstract (arXiv)

We propose the concept of a Lagrangian game to solve constrained Markov games. Such games model scenarios where agents face cost constraints in addition to their individual rewards, that depend on both agent joint actions and the evolving environment state over time. Constrained Markov games form the formal mechanism behind safe multiagent reinforcement learning, providing a structured model for dynamic multiagent interactions in a multitude of settings, such as autonomous teams operating under local energy and time constraints, for example. We develop a primal-dual approach in which agents solve a Lagrangian game associated with the current Lagrange multiplier, simulate cost and reward trajectories over a fixed horizon, and update the multiplier using accrued experience. This update rule generates a new Lagrangian game, initiating the next iteration. Our key result consists in showing that the sequence of solutions to these Lagrangian games yields a nonstationary Nash solution for the original constrained Markov game.