# A Scalable Multi-Robot Framework for Decentralized and Asynchronous Perception-Action-Communication Loops

arXiv: https://arxiv.org/abs/2309.10164 · HTML: https://mlsanity.com/paper/2309.10164/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 36 references that are in the map.
- First posted on arXiv 18 Sep 2023.

## Builds on (most cited first, 17 of 17)

- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering](https://mlsanity.com/paper/1606.09375/) (2016)
- [Spectral Networks and Locally Connected Networks on Graphs](https://mlsanity.com/paper/1312.6203/) (2013)
- [Robot Operating System 2: Design, Architecture, and Uses In The Wild](https://mlsanity.com/paper/2211.07752/) (2022)
- [Graph Neural Networks for Decentralized Multi-Robot Path Planning](https://mlsanity.com/paper/1912.06095/) (2019)
- [Graph Neural Networks: Architectures, Stability and Transferability](https://mlsanity.com/paper/2008.01767/) (2020)
- [Learning Decentralized Controllers for Robot Swarms with Graph Neural Networks](https://mlsanity.com/paper/1903.10527/) (2019)
- [TrackMPNN: A Message Passing Graph Neural Architecture for Multi-Object Tracking](https://mlsanity.com/paper/2101.04206/) (2021)
- [Multi-Robot Coverage and Exploration using Spatial Graph Neural Networks](https://mlsanity.com/paper/2011.01119/) (2020)
- [Graph Neural Networks for Decentralized Multi-Robot Submodular Action Selection](https://mlsanity.com/paper/2105.08601/) (2021)
- [Coverage Control in Multi-Robot Systems via Graph Neural Networks](https://mlsanity.com/paper/2109.15278/) (2021)
- [Synthesizing Decentralized Controllers with Graph Neural Networks and Imitation Learning](https://mlsanity.com/paper/2012.14906/) (2020)
- [Graph Neural Networks for Multi-Robot Active Information Acquisition](https://mlsanity.com/paper/2209.12091/) (2022)
- [A Framework for Real-World Multi-Robot Systems Running Decentralized GNN-Based Policies](https://mlsanity.com/paper/2111.01777/) (2021)
- [Scalable Perception-Action-Communication Loops with Convolutional and Graph Neural Networks](https://mlsanity.com/paper/2106.13358/) (2021)
- [Comparison of Middlewares in Edge-to-Edge and Edge-to-Cloud Communication for Distributed ROS2 Systems](https://mlsanity.com/paper/2309.07496/) (2023)

## Built on by (most cited first, 1 of 1)

- [LPAC: Learnable Perception-Action-Communication Loops with Applications to Coverage Control](https://mlsanity.com/paper/2401.04855/) (2024)

## Abstract (arXiv)

We develop a decentralized Perception-Action-Communication (PAC) system for multi-robot teams that enables them to collaborate in large scale, outdoor environments. Our system natively supports deployments at any scale by leveraging a graph neural network (GNN) to diffuse information hop-by-hop across the fleet's network. This achieves global collaboration from individual robots limited to local sensing and communication. Fully asynchronous, the core modules of PAC: perception, inter-robot communication, message aggregation and action are clocked at different frequencies with information flowing between them through buffers. We implement the PAC system as a series of highly extensible ROS2 nodes to serve as the foundational infrastructure for deployable swarm systems. PAC is validated in the real world with outdoor experiments with up to N=20 quadrotor robots and in simulations based on real-world data with up to N=100. These validations show that our system upholds crucial properties for field-deployable robot collectives: scalability, resiliency and repeatability.