# Inverse reinforcement learning for indefinite mean-field social optimization with multiplicative noise

arXiv: https://arxiv.org/abs/2608.07252 · HTML: https://mlsanity.com/paper/2608.07252/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods.
- Builds on 6 of its 38 references that are in the map.
- First posted on arXiv 7 Aug 2026.

## Placed in

- Deep reinforcement learning methods / [RL for robot locomotion and manipulation](https://mlsanity.com/deeprl/rl-for-robot-locomotion-and-manipulation/)

## Builds on (most cited first, 6 of 6)

- [Social Optima in Robust Mean Field LQG Control: From Finite to Infinite Horizon](https://mlsanity.com/paper/1908.01122/) (2019)
- [Indefinite Linear Quadratic Mean Field Social Control Problems with Multiplicative Noise](https://mlsanity.com/paper/2009.09864/) (2020)
- [Inverse reinforcement learning by expert imitation for the stochastic linear-quadratic optimal control problem](https://mlsanity.com/paper/2405.17085/) (2024)
- [Mean Field Production Output Control with Sticky Prices: Nash and Social Solutions](https://mlsanity.com/paper/1810.00137/) (2018)
- [Social Optima in Mean Field Linear-Quadratic-Gaussian Control with Volatility Uncertainty](https://mlsanity.com/paper/1912.06371/) (2019)
- [Comparison of two mean-field approaches to modeling an epidemic spread](https://mlsanity.com/paper/2411.02800/) (2024)

## Abstract (arXiv)

This paper studies the inverse reinforcement learning (RL) problem for linear-quadratic mean-field (MF) social optimization. The considered system features multiplicative noise and indefinite cost weights, which violate standard convexity assumptions and pose analytical challenges. The goal is to recover unknown social cost weights from expert demonstrations and reproduce the optimal control policies. This requires solving coupled stochastic algebraic Riccati equations and Lyapunov equations with unknown system dynamics. To this end, we first propose a model-based inverse RL algorithm with two sequential loops that separately handle individual and MF dynamics, and we prove its convergence and closed-loop stabilizability. Moreover, we characterize the non-uniqueness of the recovered cost weights. To eliminate reliance on system dynamics, we develop a model-free inverse RL algorithm using integral RL and least-squares identification, which requires only measured trajectory data satisfying mild rank conditions. Finally, numerical simulations validate the effectiveness of the proposed approaches.