# Assume-Guarantee Reinforcement Learning

arXiv: https://arxiv.org/abs/2312.09938 · HTML: https://mlsanity.com/paper/2312.09938/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 40 references that are in the map.
- First posted on arXiv 15 Dec 2023.

## Builds on (most cited first, 9 of 9)

- [Compositional Reinforcement Learning from Logical Specifications](https://mlsanity.com/paper/2106.13906/) (2021)
- [Multi-Agent Reinforcement Learning with Temporal Logic Specifications](https://mlsanity.com/paper/2102.00582/) (2021)
- [Formal Controller Synthesis for Continuous-Space MDPs via Model-Free Reinforcement Learning](https://mlsanity.com/paper/2003.00712/) (2020)
- [Scalable Planning and Learning for Multiagent POMDPs: Extended Version](https://mlsanity.com/paper/1404.1140/) (2014)
- [A Decentralized Energy-Optimal Control Framework for Connected Automated Vehicles at Signal-Free Intersections](https://mlsanity.com/paper/1602.03786/) (2016)
- [Compositional abstraction and safety synthesis using overlapping symbolic models](https://mlsanity.com/paper/1704.07124/) (2017)
- [Compositional Construction of Infinite Abstractions for Networks of Stochastic Control Systems](https://mlsanity.com/paper/1801.10505/) (2018)
- [Compositional Set Invariance in Network Systems with Assume-Guarantee Contracts](https://mlsanity.com/paper/1810.10636/) (2018)
- [SAT-MARL: Specification Aware Training in Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2012.07949/) (2020)

## Built on by (most cited first, 4 of 4)

- [Data-Driven Abstractions via Binary-Tree Gaussian Processes for Formal Verification](https://mlsanity.com/paper/2407.21029/) (2024)
- [Formal Control for Uncertain Systems via Contract-Based Probabilistic Surrogates (Extended Version)](https://mlsanity.com/paper/2506.16971/) (2025)
- [Average Reward Reinforcement Learning for Omega-Regular and Mean-Payoff Objectives](https://mlsanity.com/paper/2505.15693/) (2025)
- [Kernel-Based Learning of Safety Barriers](https://mlsanity.com/paper/2601.12002/) (2026)

## Abstract (arXiv)

We present a modular approach to reinforcement learning (RL) in environments consisting of simpler components evolving in parallel. A monolithic view of such modular environments may be prohibitively large to learn, or may require unrealizable communication between the components in the form of a centralized controller. Our proposed approach is based on the assume-guarantee paradigm where the optimal control for the individual components is synthesized in isolation by making assumptions about the behaviors of neighboring components, and providing guarantees about their own behavior. We express these assume-guarantee contracts as regular languages and provide automatic translations to scalar rewards to be used in RL. By combining local probabilities of satisfaction for each component, we provide a lower bound on the probability of satisfaction of the complete system. By solving a Markov game for each component, RL can produce a controller for each component that maximizes this lower bound. The controller utilizes the information it receives through communication, observations, and any knowledge of a coarse model of other agents. We experimentally demonstrate the efficiency of the proposed approach on a variety of case studies.