# Federated Transfer Reinforcement Learning for Autonomous Driving

arXiv: https://arxiv.org/abs/1910.06001 · HTML: https://mlsanity.com/paper/1910.06001/

## Key facts

- Cited by 44 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Oct 2019.

## Built on by (most cited first, 30 of 44)

- [Subgraph Federated Learning with Missing Neighbor Generation](https://mlsanity.com/paper/2106.13430/) (2021)
- [Federated Reinforcement Learning: Techniques, Applications, and Open Challenges](https://mlsanity.com/paper/2108.11887/) (2021)
- [Fault-Tolerant Federated Reinforcement Learning with Theoretical Guarantee](https://mlsanity.com/paper/2110.14074/) (2021)
- [Federated Learning for Computationally-Constrained Heterogeneous Devices: A Survey](https://mlsanity.com/paper/2307.09182/) (2023)
- [Finite-Time Analysis of On-Policy Heterogeneous Federated Reinforcement Learning](https://mlsanity.com/paper/2401.15273/) (2024)
- [FedKL: Tackling Data Heterogeneity in Federated Reinforcement Learning by Penalizing KL Divergence](https://mlsanity.com/paper/2204.08125/) (2022)
- [Locally Estimated Global Perturbations are Better than Local Perturbations for Federated Sharpness-aware Minimization](https://mlsanity.com/paper/2405.18890/) (2024)
- [GradientCoin: A Peer-to-Peer Decentralized Large Language Models](https://mlsanity.com/paper/2308.10502/) (2023)
- [Federated Adversarial Learning: A Framework with Convergence Analysis](https://mlsanity.com/paper/2208.03635/) (2022)
- [Provably Efficient Multi-Task Reinforcement Learning with Model Transfer](https://mlsanity.com/paper/2107.08622/) (2021)
- [FedSkip: Combatting Statistical Heterogeneity with Federated Skip Aggregation](https://mlsanity.com/paper/2212.07224/) (2022)
- [FL-NTK: A Neural Tangent Kernel-based Framework for Federated Learning Convergence Analysis](https://mlsanity.com/paper/2105.05001/) (2021)
- [Failure Prediction in Production Line Based on Federated Learning: An Empirical Study](https://mlsanity.com/paper/2101.11715/) (2021)
- [A Survey on Federated Learning and its Applications for Accelerating Industrial Internet of Things](https://mlsanity.com/paper/2104.10501/) (2021)
- [Deep Transfer Learning for Intelligent Vehicle Perception: a Survey](https://mlsanity.com/paper/2306.15110/) (2023)
- [Federated Empirical Risk Minimization via Second-Order Method](https://mlsanity.com/paper/2305.17482/) (2023)
- [Handling Data Heterogeneity via Architectural Design for Federated Visual Recognition](https://mlsanity.com/paper/2310.15165/) (2023)
- [Communication Resources Constrained Hierarchical Federated Learning for End-to-End Autonomous Driving](https://mlsanity.com/paper/2306.16169/) (2023)
- [Addressing modern and practical challenges in machine learning: A survey of online federated and transfer learning](https://mlsanity.com/paper/2202.03070/) (2022)
- [FedHQL: Federated Heterogeneous Q-Learning](https://mlsanity.com/paper/2301.11135/) (2023)
- [Local Environment Poisoning Attacks on Federated Reinforcement Learning](https://mlsanity.com/paper/2303.02725/) (2023)
- [The Gradient Convergence Bound of Federated Multi-Agent Reinforcement Learning with Efficient Communication](https://mlsanity.com/paper/2103.13026/) (2021)
- [FOVA: Offline Federated Reinforcement Learning with Mixed-Quality Data](https://mlsanity.com/paper/2512.02350/) (2025)
- [Federated Learning for 6G: Paradigms, Taxonomy, Recent Advances and Insights](https://mlsanity.com/paper/2312.04688/) (2023)
- [Federated Learning under Partially Class-Disjoint Data via Manifold Reshaping](https://mlsanity.com/paper/2405.18983/) (2024)
- [Client Selection for Federated Policy Optimization with Environment Heterogeneity](https://mlsanity.com/paper/2305.10978/) (2023)
- [Combining Federated Learning and Control: A Survey](https://mlsanity.com/paper/2407.11069/) (2024)
- [Towards Sample-Efficiency and Generalization of Transfer and Inverse Reinforcement Learning: A Comprehensive Literature Review](https://mlsanity.com/paper/2411.10268/) (2024)
- [Federated Learning over a Wireless Network: Distributed User Selection through Random Access](https://mlsanity.com/paper/2307.03758/) (2023)
- [AVDDPG: Federated reinforcement learning applied to autonomous platoon control](https://mlsanity.com/paper/2207.03484/) (2022)

## Abstract (arXiv)

Reinforcement learning (RL) is widely used in autonomous driving tasks and training RL models typically involves in a multi-step process: pre-training RL models on simulators, uploading the pre-trained model to real-life robots, and fine-tuning the weight parameters on robot vehicles. This sequential process is extremely time-consuming and more importantly, knowledge from the fine-tuned model stays local and can not be re-used or leveraged collaboratively. To tackle this problem, we present an online federated RL transfer process for real-time knowledge extraction where all the participant agents make corresponding actions with the knowledge learned by others, even when they are acting in very different environments. To validate the effectiveness of the proposed approach, we constructed a real-life collision avoidance system with Microsoft Airsim simulator and NVIDIA JetsonTX2 car agents, which cooperatively learn from scratch to avoid collisions in indoor environment with obstacle objects. We demonstrate that with the proposed framework, the simulator car agents can transfer knowledge to the RC cars in real-time, with 27% increase in the average distance with obstacles and 42% decrease in the collision counts.