mlsanityData to 14 Oct 2026

Hierarchical Reinforcement Learning with Low-Level MPC for Multi-Agent Control

arXiv 2509.15799 ↗First posted 19 Sep 2025eess.SY cs.AI cs.ROcited by 0

Not on the map. Why?

Abstract (arXiv)

Achieving safe and coordinated behavior in dynamic, constraint-rich environments remains a major challenge for learning-based control. Pure end-to-end learning often suffers from poor sample efficiency and limited reliability, while model-based methods depend on predefined references and struggle to generalize. We propose a hierarchical framework that combines tactical decision-making via reinforcement learning (RL) with low-level execution through Model Predictive Control (MPC). For the case of multi-agent systems this means that high-level policies select abstract targets from structured regions of interest (ROIs), while MPC ensures dynamically feasible and safe motion. Tested on a predator-prey benchmark, our approach outperforms end-to-end and shielding-based RL baselines in terms of reward, safety, and consistency, underscoring the benefits of combining structured learning with model-based control.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 15 of its 26 references that are in the map. First posted on arXiv 19 Sep 2025.

Builds on

15 of 26 references.

References · 15

  1. 2017Safe Model-based Reinforcement Learning with Stability Guaranteescited by 366
  2. 2018A predictive safety filter for learning-based control of constrained nonlinear dynamical systemscited by 118
  3. 2019Safe Reinforcement Learning Using Robust MPCcited by 85
  4. 2023State-wise Safe Reinforcement Learning: A Surveycited by 55
  5. 2019Robust Model Predictive Shielding for Safe Reinforcement Learning with Stochastic Dynamicscited by 47
  6. 2025Synthesis of Model Predictive Control and Reinforcement Learning: Survey and Classificationcited by 26
  7. 2022MARLlib: A Scalable and Efficient Multi-agent Reinforcement Learning Librarycited by 21
  8. 2019Learning-based Model Predictive Control for Safe Exploration and Reinforcement Learningcited by 16
  9. 2020Learning High-Level Policies for Model Predictive Controlcited by 10
  10. 2022A Hierarchical Approach for Strategic Motion Planning in Autonomous Racingcited by 7
  11. 2023Safe Multi-Agent Reinforcement Learning for Behavior-Based Cooperative Navigationcited by 6
  12. 2023Safety Guaranteed Robust Multi-Agent Reinforcement Learning with Hierarchical Control for Connected and Automated Vehiclescited by 5
3 more
  1. 2013On feasibility, stability and performance in distributed model predictive controlcited by 2
  2. 2024DeepSafeMPC: Deep Learning-Based Model Predictive Control for Safe Multi-Agent Reinforcement Learningcited by 2
  3. 2025A Provable Approach for End-to-End Safe Reinforcement Learningcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025MPCritic: A plug-and-play MPC architecture for reinforcement learning
  2. 2026Hierarchical Decision Making with Structured Policies: A Principled Design via Inverse Optimization
  3. 2025MPC4RL -- A Software Package for Reinforcement Learning based on Model Predictive Control
  4. 2026A Systematic Review and Taxonomy of Reinforcement Learning-Model Predictive Control Integration for Linear Systems
  5. 2026Merging model-based control with multi-agent reinforcement learning for multi-agent cooperative teaming strategies
  6. 2023TD-MPC2: Scalable, Robust World Models for Continuous Control
  7. 2024Subgoal-based Hierarchical Reinforcement Learning for Multi-Agent Collaboration ↗
  8. 2026Optimal Control with Learned Critics under Unmodeled State Dependencies