# Optimal Control of Logically Constrained Partially Observable and Multi-Agent Markov Decision Processes

arXiv: https://arxiv.org/abs/2305.14736 · HTML: https://mlsanity.com/paper/2305.14736/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 40 references that are in the map.
- First posted on arXiv 24 May 2023.

## Builds on (most cited first, 14 of 14)

- [Convex optimization](https://mlsanity.com/paper/2106.01946/) (2021)
- [Introduction to Online Convex Optimization](https://mlsanity.com/paper/1909.05207/) (2019)
- [Omega-Regular Objectives in Model-Free Reinforcement Learning](https://mlsanity.com/paper/1810.00950/) (2018)
- [Control Synthesis from Linear Temporal Logic Specifications using Model-Free Reinforcement Learning](https://mlsanity.com/paper/1909.07299/) (2019)
- [Multi-Agent Reinforcement Learning with Temporal Logic Specifications](https://mlsanity.com/paper/2102.00582/) (2021)
- [Symbolic LTLf Synthesis](https://mlsanity.com/paper/1705.08426/) (2017)
- [Stochastic Finite State Control of POMDPs with LTL Specifications](https://mlsanity.com/paper/2001.07679/) (2020)
- [Point-Based Methods for Model Checking in Partially Observable Markov Decision Processes](https://mlsanity.com/paper/2001.03809/) (2020)
- [Counterexample-Guided Strategy Improvement for POMDPs Using Recurrent Neural Networks](https://mlsanity.com/paper/1903.08428/) (2019)
- [Optimal Control of Partially Observable Markov Decision Processes with Finite Linear Temporal Logic Constraints](https://mlsanity.com/paper/2203.09038/) (2022)
- [DistSPECTRL: Distributing Specifications in Multi-Agent Reinforcement Learning Systems](https://mlsanity.com/paper/2206.13754/) (2022)
- [Verification and Control of Partially Observable Probabilistic Real-Time Systems](https://mlsanity.com/paper/1506.06419/) (2015)
- [Distributed Control using Reinforcement Learning with Temporal-Logic-Based Reward Shaping](https://mlsanity.com/paper/2203.04172/) (2022)
- [Optimal Communication and Control Strategies for a Multi-Agent System in the Presence of an Adversary](https://mlsanity.com/paper/2209.03888/) (2022)

## Built on by (most cited first, 2 of 2)

- [Variance-Reduced Cascade Q-learning: Algorithms and Sample Complexity](https://mlsanity.com/paper/2408.06544/) (2024)
- [Ensuring Logic in the Fog: Sound POMDP Synthesis with LTL Objectives](https://mlsanity.com/paper/2605.12581/) (2026)

## Abstract (arXiv)

Autonomous systems often have logical constraints arising, for example, from safety, operational, or regulatory requirements. Such constraints can be expressed using temporal logic specifications. The system state is often partially observable. Moreover, it could encompass a team of multiple agents with a common objective but disparate information structures and constraints. In this paper, we first introduce an optimal control theory for partially observable Markov decision processes (POMDPs) with finite linear temporal logic constraints. We provide a structured methodology for synthesizing policies that maximize a cumulative reward while ensuring that the probability of satisfying a temporal logic constraint is sufficiently high. Our approach comes with guarantees on approximate reward optimality and constraint satisfaction. We then build on this approach to design an optimal control framework for logically constrained multi-agent settings with information asymmetry. We illustrate the effectiveness of our approach by implementing it on several case studies.