# Adaptive Federated Dropout: Improving Communication Efficiency and Generalization for Federated Learning

arXiv: https://arxiv.org/abs/2011.04050 · HTML: https://mlsanity.com/paper/2011.04050/

## Key facts

- Cited by 37 papers in the map (data through 7 Oct 2026).
- First posted on arXiv 8 Nov 2020.

## Built on by (most cited first, 30 of 37)

- [FedProc: Prototypical Contrastive Federated Learning on Non-IID data](https://mlsanity.com/paper/2109.12273/) (2021)
- [Towards Fairness-Aware Federated Learning](https://mlsanity.com/paper/2111.01872/) (2021)
- [DAdaQuant: Doubly-adaptive quantization for communication-efficient Federated Learning](https://mlsanity.com/paper/2111.00465/) (2021)
- [Recent Advances on Federated Learning: A Systematic Survey](https://mlsanity.com/paper/2301.01299/) (2023)
- [No One Left Behind: Inclusive Federated Learning over Heterogeneous Devices](https://mlsanity.com/paper/2202.08036/) (2022)
- [MP-FedCL: Multiprototype Federated Contrastive Learning for Edge Intelligence](https://mlsanity.com/paper/2304.01950/) (2023)
- [ResIST: Layer-Wise Decomposition of ResNets for Distributed Training](https://mlsanity.com/paper/2107.00961/) (2021)
- [Towards Fair Federated Recommendation Learning: Characterizing the Inter-Dependence of System and Data Heterogeneity](https://mlsanity.com/paper/2206.02633/) (2022)
- [FedOBD: Opportunistic Block Dropout for Efficiently Training Large-scale Neural Networks through Federated Learning](https://mlsanity.com/paper/2208.05174/) (2022)
- [Enabling On-Device Training of Speech Recognition Models with Federated Dropout](https://mlsanity.com/paper/2110.03634/) (2021)
- [Communication-Efficient Federated Learning with Dual-Side Low-Rank Compression](https://mlsanity.com/paper/2104.12416/) (2021)
- [Adaptive Federated Pruning in Hierarchical Wireless Networks](https://mlsanity.com/paper/2305.09042/) (2023)
- [FedBAT: Communication-Efficient Federated Learning via Learnable Binarization](https://mlsanity.com/paper/2408.03215/) (2024)
- [FedDropoutAvg: Generalizable federated learning for histopathology image classification](https://mlsanity.com/paper/2111.13230/) (2021)
- [HideNseek: Federated Lottery Ticket via Server-side Pruning and Sign Supermask](https://mlsanity.com/paper/2206.04385/) (2022)
- [Efficient Federated Learning Using Dynamic Update and Adaptive Pruning with Momentum on Shared Server Data](https://mlsanity.com/paper/2408.05678/) (2024)
- [The Panaceas for Improving Low-Rank Decomposition in Communication-Efficient Federated Learning](https://mlsanity.com/paper/2505.23176/) (2025)
- [Supernet Training for Federated Image Classification under System Heterogeneity](https://mlsanity.com/paper/2206.01366/) (2022)
- [FedorAS: Federated Architecture Search under system heterogeneity](https://mlsanity.com/paper/2206.11239/) (2022)
- [Efficient Training of Large-scale Industrial Fault Diagnostic Models through Federated Opportunistic Block Dropout](https://mlsanity.com/paper/2302.11485/) (2023)
- [Federated Neural Radiance Fields](https://mlsanity.com/paper/2305.01163/) (2023)
- [FedBIAD: Communication-Efficient and Accuracy-Guaranteed Federated Learning with Bayesian Inference-Based Adaptive Dropout](https://mlsanity.com/paper/2307.07172/) (2023)
- [Exploring the Practicality of Federated Learning: A Survey Towards the Communication Perspective](https://mlsanity.com/paper/2405.20431/) (2024)
- [FedNS: Improving Federated Learning for collaborative image classification on mobile clients](https://mlsanity.com/paper/2101.07995/) (2021)
- [FedDD: Toward Communication-efficient Federated Learning with Differential Parameter Dropout](https://mlsanity.com/paper/2308.16835/) (2023)
- [Personalized federated learning based on feature fusion](https://mlsanity.com/paper/2406.16583/) (2024)
- [FedPeWS: Personalized Warmup via Subnetworks for Enhanced Heterogeneous Federated Learning](https://mlsanity.com/paper/2410.03042/) (2024)
- [Learnable Sparse Customization in Heterogeneous Edge Computing](https://mlsanity.com/paper/2412.07216/) (2024)
- [Efficient Federated Learning with Heterogeneous Data and Adaptive Dropout](https://mlsanity.com/paper/2507.10430/) (2025)
- [FedNet2Net: Saving Communication and Computations in Federated Learning with Model Growing](https://mlsanity.com/paper/2207.09568/) (2022)

## Abstract (arXiv)

With more regulations tackling users' privacy-sensitive data protection in recent years, access to such data has become increasingly restricted and controversial. To exploit the wealth of data generated and located at distributed entities such as mobile phones, a revolutionary decentralized machine learning setting, known as Federated Learning, enables multiple clients located at different geographical locations to collaboratively learn a machine learning model while keeping all their data on-device. However, the scale and decentralization of federated learning present new challenges. Communication between the clients and the server is considered a main bottleneck in the convergence time of federated learning. In this paper, we propose and study Adaptive Federated Dropout (AFD), a novel technique to reduce the communication costs associated with federated learning. It optimizes both server-client communications and computation costs by allowing clients to train locally on a selected subset of the global model. We empirically show that this strategy, combined with existing compression methods, collectively provides up to 57x reduction in convergence time. It also outperforms the state-of-the-art solutions for communication efficiency. Furthermore, it improves model generalization by up to 1.7%.