# Deep Federated Learning for Autonomous Driving

arXiv: https://arxiv.org/abs/2110.05754 · HTML: https://mlsanity.com/paper/2110.05754/

## Key facts

- Cited by 54 papers in the map (data through 14 Oct 2026).
- Placed in End-to-end system architecture, one of 9 directions in End-to-end autonomous driving.
- Builds on 43 of its 63 references that are in the map.
- First posted on arXiv 12 Oct 2021.

## Placed in

- End-to-end autonomous driving / [End-to-end system architecture](https://mlsanity.com/driving/end-to-end-system-architecture/)

## Builds on (most cited first, 30 of 43)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [Advances and Open Problems in Federated Learning](https://mlsanity.com/paper/1912.04977/) (2019)
- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [Federated Learning: Challenges, Methods, and Future Directions](https://mlsanity.com/paper/1908.07873/) (2019)
- [End to End Learning for Self-Driving Cars](https://mlsanity.com/paper/1604.07316/) (2016)
- [The Replica Dataset: A Digital Replica of Indoor Spaces](https://mlsanity.com/paper/1906.05797/) (2019)
- [Network In Network](https://mlsanity.com/paper/1312.4400/) (2013)
- [Can Decentralized Algorithms Outperform Centralized Algorithms? A Case Study for Decentralized Parallel Stochastic Gradient Descent](https://mlsanity.com/paper/1705.09056/) (2017)
- [Robust and Communication-Efficient Federated Learning from Non-IID Data](https://mlsanity.com/paper/1903.02891/) (2019)
- [Multi-Modal Fusion Transformer for End-to-End Autonomous Driving](https://mlsanity.com/paper/2104.09224/) (2021)
- [Learning to Navigate in Complex Environments](https://mlsanity.com/paper/1611.03673/) (2016)
- [End-to-end Learning of Driving Models from Large-scale Video Datasets](https://mlsanity.com/paper/1612.01079/) (2016)
- [Cooperative SGD: A unified Framework for the Design and Analysis of Communication-Efficient SGD Algorithms](https://mlsanity.com/paper/1808.07576/) (2018)
- [Learning to Learn How to Learn: Self-Adaptive Visual Navigation Using Meta-Learning](https://mlsanity.com/paper/1812.00971/) (2018)
- [Ground-aware Monocular 3D Object Detection for Autonomous Driving](https://mlsanity.com/paper/2102.00690/) (2021)
- [Deep Reinforcement Learning with Successor Features for Navigation across Similar Environments](https://mlsanity.com/paper/1612.05533/) (2016)
- [From Perception to Decision: A Data-driven Approach to End-to-end Motion Planning for Autonomous Ground Robots](https://mlsanity.com/paper/1609.07910/) (2016)
- [Decentralized Federated Learning via Mutual Knowledge Transfer](https://mlsanity.com/paper/2012.13063/) (2020)
- [MultiSiam: Self-supervised Multi-instance Siamese Representation Learning for Autonomous Driving](https://mlsanity.com/paper/2108.12178/) (2021)
- [Throughput-Optimal Topology Design for Cross-Silo Federated Learning](https://mlsanity.com/paper/2010.12229/) (2020)
- [Variational End-to-End Navigation and Localization](https://mlsanity.com/paper/1811.10119/) (2018)
- [Shared Cross-Modal Trajectory Prediction for Autonomous Driving](https://mlsanity.com/paper/2011.08436/) (2020)
- [Coarse-to-Fine Reasoning for Visual Question Answering](https://mlsanity.com/paper/2110.02526/) (2021)
- [Distributed Dynamic Map Fusion via Federated Learning for Intelligent Networked Vehicles](https://mlsanity.com/paper/2103.03786/) (2021)
- [Binary TTC: A Temporal Geofence for Autonomous Navigation](https://mlsanity.com/paper/2101.04777/) (2021)
- [Toward Low-Flying Autonomous MAV Trail Navigation using Deep Neural Networks for Environmental Awareness](https://mlsanity.com/paper/1705.02550/) (2017)

## Built on by (most cited first, 30 of 54)

- [Federated Learning for Generalization, Robustness, Fairness: A Survey and Benchmark](https://mlsanity.com/paper/2311.06750/) (2023)
- [Decentralized Federated Learning: A Survey and Perspective](https://mlsanity.com/paper/2306.01603/) (2023)
- [FedDrive: Generalizing Federated Learning to Semantic Segmentation in Autonomous Driving](https://mlsanity.com/paper/2202.13670/) (2022)
- [Federated Learning for Connected and Automated Vehicles: A Survey of Existing Approaches and Challenges](https://mlsanity.com/paper/2308.10407/) (2023)
- [Exploring One-shot Semi-supervised Federated Learning with A Pre-trained Diffusion Model](https://mlsanity.com/paper/2305.04063/) (2023)
- [Topology-aware Federated Learning in Edge Computing: A Comprehensive Survey](https://mlsanity.com/paper/2302.02573/) (2023)
- [Federated Learning for Computer Vision](https://mlsanity.com/paper/2308.13558/) (2023)
- [Peer-to-Peer Federated Continual Learning for Naturalistic Driving Action Recognition](https://mlsanity.com/paper/2304.07421/) (2023)
- [Collaborative Learning of Anomalies with Privacy (CLAP) for Unsupervised Video Anomaly Detection: A New Baseline](https://mlsanity.com/paper/2404.00847/) (2024)
- [FedBEVT: Federated Learning Bird's Eye View Perception Transformer in Road Traffic Systems](https://mlsanity.com/paper/2304.01534/) (2023)
- [FedZero: Leveraging Renewable Excess Energy in Federated Learning](https://mlsanity.com/paper/2305.15092/) (2023)
- [FedWon: Triumphing Multi-domain Federated Learning Without Normalization](https://mlsanity.com/paper/2306.05879/) (2023)
- [Optimal Federated Learning for Nonparametric Regression with Heterogeneous Distributed Differential Privacy Constraints](https://mlsanity.com/paper/2406.06755/) (2024)
- [HabiCrowd: A High Performance Simulator for Crowd-Aware Visual Navigation](https://mlsanity.com/paper/2306.11377/) (2023)
- [A Survey on Vulnerability of Federated Learning: A Learning Algorithm Perspective](https://mlsanity.com/paper/2311.16065/) (2023)
- [Fine-Grained Visual Classification using Self Assessment Classifier](https://mlsanity.com/paper/2205.10529/) (2022)
- [Communication Resources Constrained Hierarchical Federated Learning for End-to-End Autonomous Driving](https://mlsanity.com/paper/2306.16169/) (2023)
- [Federated Online and Bandit Convex Optimization](https://mlsanity.com/paper/2311.17586/) (2023)
- [Continual Adaptation of Vision Transformers for Federated Learning](https://mlsanity.com/paper/2306.09970/) (2023)
- [Federated Nonparametric Hypothesis Testing with Differential Privacy Constraints: Optimal Rates and Adaptive Tests](https://mlsanity.com/paper/2406.06749/) (2024)
- [One-Shot Heterogeneous Federated Learning with Local Model-Guided Diffusion Models](https://mlsanity.com/paper/2311.08870/) (2023)
- [On the Effect of Defections in Federated Learning and How to Prevent Them](https://mlsanity.com/paper/2311.16459/) (2023)
- [Exploring the Practicality of Federated Learning: A Survey Towards the Communication Perspective](https://mlsanity.com/paper/2405.20431/) (2024)
- [FedLF: Adaptive Logit Adjustment and Feature Optimization in Federated Long-Tailed Learning](https://mlsanity.com/paper/2409.12105/) (2024)
- [Fed-EC: Bandwidth-Efficient Clustering-Based Federated Learning For Autonomous Visual Robot Navigation](https://mlsanity.com/paper/2411.04112/) (2024)
- [Emerging Threats in Deep Learning-Based Autonomous Driving: A Comprehensive Survey](https://mlsanity.com/paper/2210.11237/) (2022)
- [Federated clustering with GAN-based data synthesis](https://mlsanity.com/paper/2210.16524/) (2022)
- [Decentralized Federated Learning with Gradient Tracking over Time-Varying Directed Networks](https://mlsanity.com/paper/2409.17189/) (2024)
- [Optimizing Federated Learning by Entropy-Based Client Selection](https://mlsanity.com/paper/2411.01240/) (2024)
- [Bayesian Deep Learning Via Expectation Maximization and Turbo Deep Approximate Message Passing](https://mlsanity.com/paper/2402.07366/) (2024)

## Abstract (arXiv)

Autonomous driving is an active research topic in both academia and industry. However, most of the existing solutions focus on improving the accuracy by training learnable models with centralized large-scale data. Therefore, these methods do not take into account the user's privacy. In this paper, we present a new approach to learn autonomous driving policy while respecting privacy concerns. We propose a peer-to-peer Deep Federated Learning (DFL) approach to train deep architectures in a fully decentralized manner and remove the need for central orchestration. We design a new Federated Autonomous Driving network (FADNet) that can improve the model stability, ensure convergence, and handle imbalanced data distribution problems while is being trained with federated learning methods. Intensively experimental results on three datasets show that our approach with FADNet and DFL achieves superior accuracy compared with other recent methods. Furthermore, our approach can maintain privacy by not collecting user data to a central server.