# Lifelong Federated Reinforcement Learning: A Learning Architecture for Navigation in Cloud Robotic Systems

arXiv: https://arxiv.org/abs/1901.06455 · HTML: https://mlsanity.com/paper/1901.06455/

## Key facts

- Cited by 78 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Jan 2019.

## Built on by (most cited first, 30 of 78)

- [Federated Learning on Non-IID Data: A Survey](https://mlsanity.com/paper/2106.06843/) (2021)
- [Federated Reinforcement Learning: Techniques, Applications, and Open Challenges](https://mlsanity.com/paper/2108.11887/) (2021)
- [Federated Reinforcement Learning with Environment Heterogeneity](https://mlsanity.com/paper/2204.02634/) (2022)
- [Fault-Tolerant Federated Reinforcement Learning with Theoretical Guarantee](https://mlsanity.com/paper/2110.14074/) (2021)
- [Federated Learning for Computationally-Constrained Heterogeneous Devices: A Survey](https://mlsanity.com/paper/2307.09182/) (2023)
- [Edge-Cloud Polarization and Collaboration: A Comprehensive Survey for AI](https://mlsanity.com/paper/2111.06061/) (2021)
- [Finite-Time Analysis of On-Policy Heterogeneous Federated Reinforcement Learning](https://mlsanity.com/paper/2401.15273/) (2024)
- [FedKL: Tackling Data Heterogeneity in Federated Reinforcement Learning by Penalizing KL Divergence](https://mlsanity.com/paper/2204.08125/) (2022)
- [Federated Temporal Difference Learning with Linear Function Approximation under Environmental Heterogeneity](https://mlsanity.com/paper/2302.02212/) (2023)
- [Federated Domain Generalization: A Survey](https://mlsanity.com/paper/2306.01334/) (2023)
- [Federated Imitation Learning: A Novel Framework for Cloud Robotic Systems with Heterogeneous Sensor Data](https://mlsanity.com/paper/1912.12204/) (2019)
- [Distributed Dynamic Map Fusion via Federated Learning for Intelligent Networked Vehicles](https://mlsanity.com/paper/2103.03786/) (2021)
- [Provably Efficient Cooperative Multi-Agent Reinforcement Learning with Function Approximation](https://mlsanity.com/paper/2103.04972/) (2021)
- [Unleashing the Power of Continual Learning on Non-Centralized Devices: A Survey](https://mlsanity.com/paper/2412.13840/) (2024)
- [A Survey of Continual Reinforcement Learning](https://mlsanity.com/paper/2506.21872/) (2025)
- [Applications of Federated Learning in Smart Cities: Recent Advances, Taxonomy, and Open Challenges](https://mlsanity.com/paper/2102.01375/) (2021)
- [Provably Efficient Multi-Task Reinforcement Learning with Model Transfer](https://mlsanity.com/paper/2107.08622/) (2021)
- [Decentralized Deep Learning for Multi-Access Edge Computing: A Survey on Communication Efficiency and Trustworthiness](https://mlsanity.com/paper/2108.03980/) (2021)
- [Decentralized Unsupervised Learning of Visual Representations](https://mlsanity.com/paper/2111.10763/) (2021)
- [Multi-Task Federated Reinforcement Learning with Adversaries](https://mlsanity.com/paper/2103.06473/) (2021)
- [FedRLHF: A Convergence-Guaranteed Federated Framework for Privacy-Preserving and Personalized RLHF](https://mlsanity.com/paper/2412.15538/) (2024)
- [A Survey on Federated Learning and its Applications for Accelerating Industrial Internet of Things](https://mlsanity.com/paper/2104.10501/) (2021)
- [DISTREAL: Distributed Resource-Aware Learning in Heterogeneous Systems](https://mlsanity.com/paper/2112.08761/) (2021)
- [Cooperative Multi-Agent Reinforcement Learning: Asynchronous Communication and Linear Function Approximation](https://mlsanity.com/paper/2305.06446/) (2023)
- [Federated Learning for Privacy-Preserving Open Innovation Future on Digital Health](https://mlsanity.com/paper/2108.10761/) (2021)
- [Momentum for the Win: Collaborative Federated Reinforcement Learning across Heterogeneous Environments](https://mlsanity.com/paper/2405.19499/) (2024)
- [Provably Robust Federated Reinforcement Learning](https://mlsanity.com/paper/2502.08123/) (2025)
- [User profile-driven large-scale multi-agent learning from demonstration in federated human-robot collaborative environments](https://mlsanity.com/paper/2103.16434/) (2021)
- [Federated Reinforcement Learning for Collective Navigation of Robotic Swarms](https://mlsanity.com/paper/2202.01141/) (2022)
- [Aggregating Capacity in FL through Successive Layer Training for Computationally-Constrained Devices](https://mlsanity.com/paper/2305.17005/) (2023)

## Abstract (arXiv)

This paper was motivated by the problem of how to make robots fuse and transfer their experience so that they can effectively use prior knowledge and quickly adapt to new environments. To address the problem, we present a learning architecture for navigation in cloud robotic systems: Lifelong Federated Reinforcement Learning (LFRL). In the work, We propose a knowledge fusion algorithm for upgrading a shared model deployed on the cloud. Then, effective transfer learning methods in LFRL are introduced. LFRL is consistent with human cognitive science and fits well in cloud robotic systems. Experiments show that LFRL greatly improves the efficiency of reinforcement learning for robot navigation. The cloud robotic system deployment also shows that LFRL is capable of fusing prior knowledge. In addition, we release a cloud robotic navigation-learning website based on LFRL.