# Steering No-Regret Agents in MFGs under Model Uncertainty

arXiv: https://arxiv.org/abs/2503.09309 · HTML: https://mlsanity.com/paper/2503.09309/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in RL for games and combinatorial optimization, one of 15 directions in Deep reinforcement learning methods.
- Builds on 28 of its 63 references that are in the map.
- First posted on arXiv 12 Mar 2025.

## Placed in

- Deep reinforcement learning methods / [RL for games and combinatorial optimization](https://mlsanity.com/deeprl/rl-for-games-and-combinatorial-optimization/)

## Builds on (most cited first, 28 of 28)

- [Introduction to Online Convex Optimization](https://mlsanity.com/paper/1909.05207/) (2019)
- [Model-based Reinforcement Learning and the Eluder Dimension](https://mlsanity.com/paper/1406.1853/) (2014)
- [Online Convex Optimization in Adversarial Markov Decision Processes](https://mlsanity.com/paper/1905.07773/) (2019)
- [Learning in Mean Field Games: A Survey](https://mlsanity.com/paper/2205.12944/) (2022)
- [Strategizing against No-regret Learners](https://mlsanity.com/paper/1909.13861/) (2019)
- [Adaptive Contract Design for Crowdsourcing Markets: Bandit Algorithms for Repeated Principal-Agent Problems](https://mlsanity.com/paper/1405.2875/) (2014)
- [The Sample Complexity of Online Contract Design](https://mlsanity.com/paper/2211.05732/) (2022)
- [Scaling up Mean Field Games with Online Mirror Descent](https://mlsanity.com/paper/2103.00623/) (2021)
- [Policy Mirror Ascent for Efficient and Independent Learning in Mean Field Games](https://mlsanity.com/paper/2212.14449/) (2022)
- [Is Learning in Games Good for the Learners?](https://mlsanity.com/paper/2305.19496/) (2023)
- [Adaptive Incentive Design with Multi-Agent Meta-Gradient Reinforcement Learning](https://mlsanity.com/paper/2112.10859/) (2021)
- [Mechanisms for a No-Regret Agent: Beyond the Common Prior](https://mlsanity.com/paper/2009.05518/) (2020)
- [Multi-Agent Contracts](https://mlsanity.com/paper/2211.05434/) (2022)
- [On the Statistical Efficiency of Mean-Field Reinforcement Learning with General Function Approximation](https://mlsanity.com/paper/2305.11283/) (2023)
- [Incentivized Learning in Principal-Agent Bandit Games](https://mlsanity.com/paper/2403.03811/) (2024)
- [Coordinating Followers to Reach Better Equilibria: End-to-End Gradient Descent for Stackelberg Games](https://mlsanity.com/paper/2106.03278/) (2021)
- [Adaptive Mechanism Design: Learning to Promote Cooperation](https://mlsanity.com/paper/1806.04067/) (2018)
- [Model-Based RL for Mean-Field Games is not Statistically Harder than Single-Agent RL](https://mlsanity.com/paper/2402.05724/) (2024)
- [Inducing Equilibria via Incentives: Simultaneous Design-and-Play Ensures Global Convergence](https://mlsanity.com/paper/2110.01212/) (2021)
- [A tale of a Principal and many many Agents](https://mlsanity.com/paper/1608.05226/) (2016)
- [Solving N-player dynamic routing games with congestion: a mean field approach](https://mlsanity.com/paper/2110.11943/) (2021)
- [Automated Design of Affine Maximizer Mechanisms in Dynamic Settings](https://mlsanity.com/paper/2402.08129/) (2024)
- [Multi-Agent Contract Design: How to Commission Multiple Agents with Individual Outcome](https://mlsanity.com/paper/2301.13654/) (2023)
- [Steering No-Regret Learners to a Desired Equilibrium](https://mlsanity.com/paper/2306.05221/) (2023)
- [MF-OML: Online Mean-Field Reinforcement Learning with Occupation Measures for Large Population Games](https://mlsanity.com/paper/2405.00282/) (2024)
- [Learning to Steer Markovian Agents under Model Uncertainty](https://mlsanity.com/paper/2407.10207/) (2024)
- [Incentive Designs for Stackelberg Games with a Large Number of Followers and their Mean-Field Limits](https://mlsanity.com/paper/2207.10611/) (2022)
- [MESOB: Balancing Equilibria & Social Optimality](https://mlsanity.com/paper/2307.07911/) (2023)

## Built on by (most cited first, 1 of 1)

- [Microeconomic Foundations of Multi-Agent Learning](https://mlsanity.com/paper/2601.03451/) (2026)

## Abstract (arXiv)

Incentive design is a popular framework for guiding agents' learning dynamics towards desired outcomes by providing additional payments beyond intrinsic rewards. However, most existing works focus on a finite, small set of agents or assume complete knowledge of the game, limiting their applicability to real-world scenarios involving large populations and model uncertainty. To address this gap, we study the design of steering rewards in Mean-Field Games (MFGs) with density-independent transitions, where both the transition dynamics and intrinsic reward functions are unknown. This setting presents non-trivial challenges, as the mediator must incentivize the agents to explore for its model learning under uncertainty, while simultaneously steer them to converge to desired behaviors without incurring excessive incentive payments. Assuming agents exhibit no(-adaptive) regret behaviors, we contribute novel optimistic exploration algorithms. Theoretically, we establish sub-linear regret guarantees for the cumulative gaps between the agents' behaviors and the desired ones. In terms of the steering cost, we demonstrate that our total incentive payments incur only sub-linear excess, competing with a baseline steering strategy that stabilizes the target policy as an equilibrium. Our work presents an effective framework for steering agents behaviors in large-population systems under uncertainty.