# Mean Field LQG Social Optimization: A Reinforcement Learning Approach

arXiv: https://arxiv.org/abs/2410.15119 · HTML: https://mlsanity.com/paper/2410.15119/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in RL for games and combinatorial optimization, one of 15 directions in Deep reinforcement learning methods.
- Builds on 26 of its 63 references that are in the map.
- First posted on arXiv 19 Oct 2024.

## Placed in

- Deep reinforcement learning methods / [RL for games and combinatorial optimization](https://mlsanity.com/deeprl/rl-for-games-and-combinatorial-optimization/)

## Builds on (most cited first, 26 of 26)

- [A Machine Learning Framework for Solving High-Dimensional Mean Field Game and Mean Field Control Problems](https://mlsanity.com/paper/1912.01825/) (2019)
- [Approximately Solving Mean Field Games via Entropy-Regularized Deep Reinforcement Learning](https://mlsanity.com/paper/2102.01585/) (2021)
- [Mean-Field Controls with Q-learning for Cooperative MARL: Convergence and Complexity Analysis](https://mlsanity.com/paper/2002.04131/) (2020)
- [Unified Reinforcement Q-Learning for Mean Field Game and Control Problems](https://mlsanity.com/paper/2006.13912/) (2020)
- [On the Convergence of Model Free Learning in Mean Field Games](https://mlsanity.com/paper/1907.02633/) (2019)
- [Scalable Deep Reinforcement Learning Algorithms for Mean Field Games](https://mlsanity.com/paper/2203.11973/) (2022)
- [Mean Field Games Flock! The Reinforcement Learning Way](https://mlsanity.com/paper/2105.07933/) (2021)
- [Linear-Quadratic Mean-Field Reinforcement Learning: Convergence of Policy Gradient Methods](https://mlsanity.com/paper/1910.04295/) (2019)
- [Deep Learning Methods for Mean Field Control Problems with Delay](https://mlsanity.com/paper/1905.00358/) (2019)
- [Proximal methods for stationary Mean Field Games with local couplings](https://mlsanity.com/paper/1608.07701/) (2016)
- [Stochastic Linear Quadratic Optimal Control Problem: A Reinforcement Learning Method](https://mlsanity.com/paper/2011.14516/) (2020)
- [Reinforcement Learning in Non-Stationary Discrete-Time Linear-Quadratic Mean-Field Games](https://mlsanity.com/paper/2009.04350/) (2020)
- [Convergence Analysis of Machine Learning Algorithms for the Numerical Solution of Mean Field Control and Games: I -- The Ergodic Case](https://mlsanity.com/paper/1907.05980/) (2019)
- [Policy iteration method for time-dependent Mean Field Games systems with non-separable Hamiltonians](https://mlsanity.com/paper/2110.02552/) (2021)
- [On stabilizability and exact observability of stochastic systems with their applications](https://mlsanity.com/paper/2307.08915/) (2023)
- [Social Optima in Robust Mean Field LQG Control: From Finite to Infinite Horizon](https://mlsanity.com/paper/1908.01122/) (2019)
- [Linear quadratic mean field social optimization: Asymptotic solvability and decentralized control](https://mlsanity.com/paper/2012.15468/) (2020)
- [Robust Mean Field Social Control Problems with Applications in Analysis of Opinion Dynamics](https://mlsanity.com/paper/2002.12040/) (2020)
- [Mean Field Linear Quadratic Control: Uniform Stabilization and Social Optimality](https://mlsanity.com/paper/2002.12363/) (2020)
- [Indefinite Linear Quadratic Mean Field Social Control Problems with Multiplicative Noise](https://mlsanity.com/paper/2009.09864/) (2020)
- [Dynamic Collective Choice: Social Optima](https://mlsanity.com/paper/1606.05272/) (2016)
- [Mean Field Production Output Control with Sticky Prices: Nash and Social Solutions](https://mlsanity.com/paper/1810.00137/) (2018)
- [Team-Optimal Solution of Finite Number of Mean-Field Coupled LQG Subsystems](https://mlsanity.com/paper/2012.02052/) (2020)
- [Rates of convergence for the policy iteration method for Mean Field Games systems](https://mlsanity.com/paper/2108.00755/) (2021)
- [Mean Field Games for Multi-agent Systems with Multiplicative Noises](https://mlsanity.com/paper/1906.03177/) (2019)
- [An Addendum to the Problem of Zero-Sum LQ Stochastic Mean-Field Dynamic Games (Extended version)](https://mlsanity.com/paper/2302.09609/) (2023)

## Built on by (most cited first, 4 of 4)

- [Robust Mean Field Social Control: A Unified Reinforcement Learning Framework](https://mlsanity.com/paper/2502.20029/) (2025)
- [Reinforcement Learning for Discrete-time LQG Mean Field Social Control Problems with Unknown Dynamics](https://mlsanity.com/paper/2507.01420/) (2025)
- [Data-Driven Network LQG Mean Field Games with Heterogeneous Populations via Integral Reinforcement Learning](https://mlsanity.com/paper/2602.14339/) (2026)
- [Decentralized Strategies for Finite Population LQG Social Control: A Reinforcement Learning Approach](https://mlsanity.com/paper/2608.30244/) (2026)

## Abstract (arXiv)

This paper presents a novel model-free method to solve linear quadratic Gaussian mean field social control problems in the presence of multiplicative noise. The objective is to achieve a social optimum by solving two algebraic Riccati equations (AREs) and determining a mean field (MF) state, both without requiring prior knowledge of individual system dynamics for all agents. In the proposed approach, we first employ integral reinforcement learning techniques to develop two model-free iterative equations that converge to solutions for the stochastic ARE and the induced indefinite ARE respectively. Then, the MF state is approximated, either through the Monte Carlo method with the obtained gain matrices or through the system identification with the measured data. Notably, a unified state and input samples collected from a single agent are used in both iterations and identification procedure, making the method more computationally efficient and scalable. Finally, a numerical example is given to demonstrate the effectiveness of the proposed algorithm.