mlsanityData to 14 Oct 2026

Second-Order MPC-Based Distributed Q-Learning

arXiv 2511.16424 ↗First posted 20 Nov 2025eess.SY cs.SYcited by 0

On the map: Deep reinforcement learning methods → RL for autonomous driving and traffic

Abstract (arXiv)

The state of the art for model predictive control (MPC)-based distributed Q-learning is limited to first-order gradient updates of the MPC parameterization. In general, using secondorder information can significantly improve the speed of convergence for learning, allowing the use of higher learning rates without introducing instability. This work presents a second-order extension to MPC-based Q-learning with updates distributed across local agents, relying only on locally available information and neighbor-to-neighbor communication. In simulation the approach is demonstrated to significantly outperform first-order distributed Q-learning.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL for autonomous driving and traffic, one of 15 directions in Deep reinforcement learning methods. Builds on 4 of its 17 references that are in the map. First posted on arXiv 20 Nov 2025.

Builds on

4 of 17 references.

References · 4

  1. 2019Data-driven Economic NMPC using Reinforcement Learningcited by 68
  2. 2024Reinforcement Learning-based Model Predictive Control for Greenhouse Climate Controlcited by 10
  3. 2023Reinforcement Learning with Model Predictive Control for Highway Ramp Meteringcited by 4
  4. 2023Multi-Agent Reinforcement Learning via Distributed MPC as a Function Approximatorcited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Scalable Reinforcement Learning for Linear-Quadratic Control of Networks
  2. 2024Faster Model Predictive Control via Self-Supervised Initialization Learning
  3. 2023IQL-TD-MPC: Implicit Q-Learning for Hierarchical Model Predictive Control
  4. 2024A finite time analysis of distributed Q-learning
  5. 2026Variance-Reduced Q-Learning over Static and Time-Varying Networks
  6. 2025Computationally efficient Gauss-Newton reinforcement learning for model predictive control
  7. 2024Toward Scalable Multirobot Control: Fast Policy Learning in Distributed MPC
  8. 2022Learning to Optimize in Model Predictive Control