# Model-based Dynamic Shielding for Safe and Efficient Multi-Agent Reinforcement Learning

arXiv: https://arxiv.org/abs/2304.06281 · HTML: https://mlsanity.com/paper/2304.06281/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 56 references that are in the map.
- First posted on arXiv 13 Apr 2023.

## Builds on (most cited first, 27 of 27)

- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments](https://mlsanity.com/paper/1706.02275/) (2017)
- [World Models](https://mlsanity.com/paper/1803.10122/) (2018)
- [Learning Latent Dynamics for Planning from Pixels](https://mlsanity.com/paper/1811.04551/) (2018)
- [Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithms](https://mlsanity.com/paper/1911.10635/) (2019)
- [Safe Reinforcement Learning via Shielding](https://mlsanity.com/paper/1708.08611/) (2017)
- [Safe, Multi-Agent, Reinforcement Learning for Autonomous Driving](https://mlsanity.com/paper/1610.03295/) (2016)
- [A Lyapunov-based Approach to Safe Reinforcement Learning](https://mlsanity.com/paper/1805.07708/) (2018)
- [End-to-End Safe Reinforcement Learning through Barrier Functions for Safety-Critical Continuous Control Tasks](https://mlsanity.com/paper/1903.08792/) (2019)
- [Recovery RL: Safe Reinforcement Learning with Learned Recovery Zones](https://mlsanity.com/paper/2010.15920/) (2020)
- [SMARTS: Scalable Multi-Agent Reinforcement Learning Training School for Autonomous Driving](https://mlsanity.com/paper/2010.09776/) (2020)
- [Safe Reinforcement Learning in Constrained Markov Decision Processes](https://mlsanity.com/paper/2008.06626/) (2020)
- [Omega-Regular Objectives in Model-Free Reinforcement Learning](https://mlsanity.com/paper/1810.00950/) (2018)
- [Control Synthesis from Linear Temporal Logic Specifications using Model-Free Reinforcement Learning](https://mlsanity.com/paper/1909.07299/) (2019)
- [Learning Safe Multi-Agent Control with Decentralized Neural Barrier Certificates](https://mlsanity.com/paper/2101.05436/) (2021)
- [Learning for Safety-Critical Control with Control Barrier Functions](https://mlsanity.com/paper/1912.10099/) (2019)
- [Safe Multi-Agent Reinforcement Learning via Shielding](https://mlsanity.com/paper/2101.11196/) (2021)
- [Safety-Critical Model Predictive Control with Discrete-Time Control Barrier Function](https://mlsanity.com/paper/2007.11718/) (2020)
- [Robust Model Predictive Shielding for Safe Reinforcement Learning with Stochastic Dynamics](https://mlsanity.com/paper/1910.10885/) (2019)
- [Cautious Reinforcement Learning with Logical Constraints](https://mlsanity.com/paper/2002.12156/) (2020)
- [Multi-Robot Collision Avoidance under Uncertainty with Probabilistic Safety Barrier Certificates](https://mlsanity.com/paper/1912.09957/) (2019)
- [Probabilistic Safety-Assured Adaptive Merging Control for Autonomous Vehicles](https://mlsanity.com/paper/2104.14159/) (2021)
- [Safe Multi-Agent Reinforcement Learning through Decentralized Multiple Control Barrier Functions](https://mlsanity.com/paper/2103.12553/) (2021)
- [Control of Mobile Robots Using Barrier Functions Under Temporal Logic Specifications](https://mlsanity.com/paper/1908.04903/) (2019)
- [Adaptive Safe Merging Control for Heterogeneous Autonomous Vehicles using Parametric Control Barrier Functions](https://mlsanity.com/paper/2202.09936/) (2022)
- [Responsibility-associated Multi-agent Collision Avoidance with Social Preferences](https://mlsanity.com/paper/2206.09030/) (2022)
- [Provable Probabilistic Safety and Feasibility-Assured Control for Autonomous Vehicles using Exponential Control Barrier Functions](https://mlsanity.com/paper/2205.03744/) (2022)

## Built on by (most cited first, 11 of 11)

- [Safe Deep Policy Adaptation](https://mlsanity.com/paper/2310.08602/) (2023)
- [Solving Multi-Agent Safe Optimal Control with Distributed Epigraph Form MARL](https://mlsanity.com/paper/2504.15425/) (2025)
- [Approximate Model-Based Shielding for Safe Reinforcement Learning](https://mlsanity.com/paper/2308.00707/) (2023)
- [Leveraging Approximate Model-based Shielding for Probabilistic Safety Guarantees in Continuous Environments](https://mlsanity.com/paper/2402.00816/) (2024)
- [Think Smart, Act SMARL! Analyzing Probabilistic Logic Shields for Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2411.04867/) (2024)
- [Adaptive GR(1) Specification Repair for Liveness-Preserving Shielding in Reinforcement Learning](https://mlsanity.com/paper/2511.02605/) (2025)
- [COOL-MC: Verifying and Explaining RL Policies for Platelet Inventory Management](https://mlsanity.com/paper/2603.02396/) (2026)
- [Mechanical Conscience: A Mathematical Framework for Dependability of Machine Intelligence](https://mlsanity.com/paper/2605.03847/) (2026)
- [Shielded Analysis: Certification and Characterization of Defensibility in Systems under Adversarial Interaction](https://mlsanity.com/paper/2606.13621/) (2026)
- [Contract-Based Compositional Shielding for Safe Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2606.14130/) (2026)
- [Decentralized Safe Multi-Agent Reinforcement Learning via Predictive Shielding](https://mlsanity.com/paper/2609.07618/) (2026)

## Abstract (arXiv)

Multi-Agent Reinforcement Learning (MARL) discovers policies that maximize reward but do not have safety guarantees during the learning and deployment phases. Although shielding with Linear Temporal Logic (LTL) is a promising formal method to ensure safety in single-agent Reinforcement Learning (RL), it results in conservative behaviors when scaling to multi-agent scenarios. Additionally, it poses computational challenges for synthesizing shields in complex multi-agent environments. This work introduces Model-based Dynamic Shielding (MBDS) to support MARL algorithm design. Our algorithm synthesizes distributive shields, which are reactive systems running in parallel with each MARL agent, to monitor and rectify unsafe behaviors. The shields can dynamically split, merge, and recompute based on agents' states. This design enables efficient synthesis of shields to monitor agents in complex environments without coordination overheads. We also propose an algorithm to synthesize shields without prior knowledge of the dynamics model. The proposed algorithm obtains an approximate world model by interacting with the environment during the early stage of exploration, making our MBDS enjoy formal safety guarantees with high probability. We demonstrate in simulations that our framework can surpass existing baselines in terms of safety guarantees and learning performance.