# Multi-Agent Reinforcement Learning via Distributed MPC as a Function Approximator

arXiv: https://arxiv.org/abs/2312.05166 · HTML: https://mlsanity.com/paper/2312.05166/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Dec 2023.

## Built on by (most cited first, 3 of 3)

- [A Conflicts-free, Speed-lossless KAN-based Reinforcement Learning Decision System for Interactive Driving in Roundabouts](https://mlsanity.com/paper/2408.08242/) (2024)
- [Second-Order MPC-Based Distributed Q-Learning](https://mlsanity.com/paper/2511.16424/) (2025)
- [Merging model-based control with multi-agent reinforcement learning for multi-agent cooperative teaming strategies](https://mlsanity.com/paper/2606.06011/) (2026)

## Abstract (arXiv)

This paper presents a novel approach to multi-agent reinforcement learning (RL) for linear systems with convex polytopic constraints. Existing work on RL has demonstrated the use of model predictive control (MPC) as a function approximator for the policy and value functions. The current paper is the first work to extend this idea to the multi-agent setting. We propose the use of a distributed MPC scheme as a function approximator, with a structure allowing for distributed learning and deployment. We then show that Q-learning updates can be performed distributively without introducing nonstationarity, by reconstructing a centralized learning update. The effectiveness of the approach is demonstrated on two numerical examples.