# Federated Reinforcement Learning for Collective Navigation of Robotic Swarms

arXiv: https://arxiv.org/abs/2202.01141 · HTML: https://mlsanity.com/paper/2202.01141/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 45 references that are in the map.
- First posted on arXiv 2 Feb 2022.

## Builds on (most cited first, 15 of 15)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Addressing Function Approximation Error in Actor-Critic Methods](https://mlsanity.com/paper/1802.09477/) (2018)
- [Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments](https://mlsanity.com/paper/1706.02275/) (2017)
- [Counterfactual Multi-Agent Policy Gradients](https://mlsanity.com/paper/1705.08926/) (2017)
- [Deep Reinforcement Learning for Autonomous Driving: A Survey](https://mlsanity.com/paper/2002.00444/) (2020)
- [Socially Aware Motion Planning with Deep Reinforcement Learning](https://mlsanity.com/paper/1703.08862/) (2017)
- [Shared Experience Actor-Critic for Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2006.07169/) (2020)
- [Lifelong Federated Reinforcement Learning: A Learning Architecture for Navigation in Cloud Robotic Systems](https://mlsanity.com/paper/1901.06455/) (2019)
- [Federated Deep Reinforcement Learning](https://mlsanity.com/paper/1901.08277/) (2019)
- [A Secure Federated Learning Framework for 5G Networks](https://mlsanity.com/paper/2005.05752/) (2020)
- [Evolution of Collective Behaviors for a Real Swarm of Aquatic Surface Robots](https://mlsanity.com/paper/1511.03154/) (2015)
- [System for multi-robotic exploration of underground environments CTU-CRAS-NORLAB in the DARPA Subterranean Challenge](https://mlsanity.com/paper/2110.05911/) (2021)
- [Secure and secret cooperation in robotic swarms](https://mlsanity.com/paper/1904.09266/) (2019)

## Built on by (most cited first, 7 of 7)

- [Fed-EC: Bandwidth-Efficient Clustering-Based Federated Learning For Autonomous Visual Robot Navigation](https://mlsanity.com/paper/2411.04112/) (2024)
- [Client Selection for Federated Policy Optimization with Environment Heterogeneity](https://mlsanity.com/paper/2305.10978/) (2023)
- [MTF-Grasp: A Multi-tier Federated Learning Approach for Robotic Grasping](https://mlsanity.com/paper/2507.10158/) (2025)
- [Sim-to-Real Deep Reinforcement Learning with Manipulators for Pick-and-place](https://mlsanity.com/paper/2309.09247/) (2023)
- [SPACE: A Python-based Simulator for Evaluating Decentralized Multi-Robot Task Allocation Algorithms](https://mlsanity.com/paper/2409.04230/) (2024)
- [Federated Learning for Large-Scale Cloud Robotic Manipulation: Opportunities and Challenges](https://mlsanity.com/paper/2507.17903/) (2025)
- [Federated Subspace Guided Vision-Language-Action Policy Distillation for Non-IID Multi-Robot Manipulation](https://mlsanity.com/paper/2609.32239/) (2026)

## Abstract (arXiv)

The recent advancement of Deep Reinforcement Learning (DRL) contributed to robotics by allowing automatic controller design. The automatic controller design is a crucial approach for designing swarm robotic systems, which require more complex controllers than a single robot system to lead a desired collective behaviour. Although the DRL-based controller design method showed its effectiveness, the reliance on the central training server is a critical problem in real-world environments where robot-server communication is unstable or limited. We propose a novel Federated Learning (FL) based DRL training strategy (FLDDPG) for use in swarm robotic applications. Through the comparison with baseline strategies under a limited communication bandwidth scenario, it is shown that the FLDDPG method resulted in higher robustness and generalisation ability into a different environment and real robots, while the baseline strategies suffer from the limitation of communication bandwidth. This result suggests that the proposed method can benefit swarm robotic systems operating in environments with limited communication bandwidth, e.g., in high-radiation, underwater, or subterranean environments.