# SafeDiSTL: Decomposing Joint Constraints for Safe Multi-Agent Reinforcement Learning

arXiv: https://arxiv.org/abs/2610.13737 · HTML: https://mlsanity.com/paper/2610.13737/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 13 of its 26 references that are in the map.
- First posted on arXiv 9 Oct 2026.

## Builds on (most cited first, 13 of 13)

- [Constrained Policy Optimization](https://mlsanity.com/paper/1705.10528/) (2017)
- [The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Games](https://mlsanity.com/paper/2103.01955/) (2021)
- [Counterfactual Multi-Agent Policy Gradients](https://mlsanity.com/paper/1705.08926/) (2017)
- [Reinforcement learning](https://mlsanity.com/paper/2405.10369/) (2024)
- [VMAS: A Vectorized Multi-Agent Simulator for Collective Robot Learning](https://mlsanity.com/paper/2207.03530/) (2022)
- [Backpropagation through Signal Temporal Logic Specifications: Infusing Logical Structure into Gradient-Based Methods](https://mlsanity.com/paper/2008.00097/) (2020)
- [Q-value Path Decomposition for Deep Multiagent Reinforcement Learning](https://mlsanity.com/paper/2002.03950/) (2020)
- [STLCG++: A Masking Approach for Differentiable Signal Temporal Logic Specification](https://mlsanity.com/paper/2501.04194/) (2025)
- [Learning Responsibility Allocations for Safe Human-Robot Interaction with Applications to Autonomous Driving](https://mlsanity.com/paper/2303.03504/) (2023)
- [Multi-Agent Reinforcement Learning Guided by Signal Temporal Logic Specifications](https://mlsanity.com/paper/2306.06808/) (2023)
- [Risk-aware Safe Control for Decentralized Multi-agent Systems via Dynamic Responsibility Allocation](https://mlsanity.com/paper/2305.13467/) (2023)
- [STL-SVPIO: Signal Temporal Logic guided Stein Variational Path Integral Optimization](https://mlsanity.com/paper/2603.13333/) (2026)
- [Differentiable Predictive Control for Large-Scale Urban Road Networks](https://mlsanity.com/paper/2406.10433/) (2024)

## Abstract (arXiv)

Cooperative multi-agent tasks often impose constraints on a shared objective or collective safety. A team level violation cost function can measure this constraint outcome, but does not distinguish which agents' actions most strongly influence that violation. We present SafeDiSTL, a constrained multi-agent reinforcement learning framework that allows for credit assignment even with joint constraint measurements. We use Signal Temporal Logic (STL) robustness to measure episode-level violation through running-maximum cost increments. Smooth full-episode robustness and differentiable dynamics provide action sensitivities, which allocate each cost increment across agents at the same decision time. These sensitivities account for delayed effects when weighting agents' shares of each cost increment. We study the framework in cooperative payload transport, connected navigation, and networked traffic control tasks. SafeDiSTL achieves 1.6-5.1x the safe-success rate in transport, improves navigation safe-success by 11.1%, and delivers 50-68% more throughput in traffic when compared to constrained baselines. Results demonstrate a strong improvement of the learned policy in dealing with shared constraint while retaining the performance of unconstrained policies.