# Formal Modelling for Multi-Robot Systems Under Uncertainty

arXiv: https://arxiv.org/abs/2305.17018 · HTML: https://mlsanity.com/paper/2305.17018/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 25 of its 125 references that are in the map.
- First posted on arXiv 26 May 2023.

## Builds on (most cited first, 25 of 25)

- [QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1803.11485/) (2018)
- [Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2003.08839/) (2020)
- [The Complexity of Decentralized Control of Markov Decision Processes](https://mlsanity.com/paper/1301.3836/) (2013)
- [FACMAC: Factored Multi-Agent Centralised Policy Gradients](https://mlsanity.com/paper/2003.06709/) (2020)
- [Learning to Teach in Cooperative Multiagent Reinforcement Learning](https://mlsanity.com/paper/1805.07830/) (2018)
- [UneVEn: Universal Value Exploration for Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2010.02974/) (2020)
- [STyLuS*: A Temporal Logic Optimal Control Synthesis Algorithm for Large-Scale Multi-Robot Systems](https://mlsanity.com/paper/1809.08345/) (2018)
- [Regularized Softmax Deep Multi-Agent Q-Learning](https://mlsanity.com/paper/2103.11883/) (2021)
- [Multi-Agent Path Finding with Delay Probabilities](https://mlsanity.com/paper/1612.05309/) (2016)
- [Macro-Action-Based Deep Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2004.08646/) (2020)
- [Asynchronous Actor-Critic for Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2209.10113/) (2022)
- [MAMBPO: Sample-efficient multi-robot reinforcement learning using learned world models](https://mlsanity.com/paper/2103.03662/) (2021)
- [Learning Multi-Robot Decentralized Macro-Action-Based Policies via a Centralized Q-Net](https://mlsanity.com/paper/1909.08776/) (2019)
- [Learning for Multi-robot Cooperation in Partially Observable Stochastic Environments with Macro-actions](https://mlsanity.com/paper/1707.07399/) (2017)
- [A Sufficient Statistic for Influence in Structured Multiagent Environments](https://mlsanity.com/paper/1907.09278/) (2019)
- [Planning for Decentralized Control of Multiple Robots Under Uncertainty](https://mlsanity.com/paper/1402.2871/) (2014)
- [Simultaneous Task Allocation and Planning Under Uncertainty](https://mlsanity.com/paper/1803.02906/) (2018)
- [Multi-agent active perception with prediction rewards](https://mlsanity.com/paper/2010.11835/) (2020)
- [MT* : Multi-Robot Path Planning for Temporal Logic Specifications](https://mlsanity.com/paper/2103.02821/) (2021)
- [Solving Transition-Independent Multi-agent MDPs with Sparse Interactions (Extended version)](https://mlsanity.com/paper/1511.09047/) (2015)
- [Near-Optimal Adversarial Policy Switching for Decentralized Asynchronous Multi-Agent Systems](https://mlsanity.com/paper/1710.06525/) (2017)
- [Local Advantage Actor-Critic for Robust Multi-Agent Deep Reinforcement Learning](https://mlsanity.com/paper/2110.08642/) (2021)
- [Semantic-level Decentralized Multi-Robot Decision-Making using Probabilistic Macro-Observations](https://mlsanity.com/paper/1703.05623/) (2017)
- [Scalable Accelerated Decentralized Multi-Robot Policy Search in Continuous Observation Spaces](https://mlsanity.com/paper/1703.05626/) (2017)
- [Probabilistic Plan Synthesis for Coupled Multi-Agent Systems](https://mlsanity.com/paper/1704.01432/) (2017)

## Abstract (arXiv)

Purpose of Review: To effectively synthesise and analyse multi-robot behaviour, we require formal task-level models which accurately capture multi-robot execution. In this paper, we review modelling formalisms for multi-robot systems under uncertainty, and discuss how they can be used for planning, reinforcement learning, model checking, and simulation. Recent Findings: Recent work has investigated models which more accurately capture multi-robot execution by considering different forms of uncertainty, such as temporal uncertainty and partial observability, and modelling the effects of robot interactions on action execution. Other strands of work have presented approaches for reducing the size of multi-robot models to admit more efficient solution methods. This can be achieved by decoupling the robots under independence assumptions, or reasoning over higher level macro actions. Summary: Existing multi-robot models demonstrate a trade off between accurately capturing robot dependencies and uncertainty, and being small enough to tractably solve real world problems. Therefore, future research should exploit realistic assumptions over multi-robot behaviour to develop smaller models which retain accurate representations of uncertainty and robot interactions; and exploit the structure of multi-robot problems, such as factored state spaces, to develop scalable solution methods.