# Federated Dropout -- A Simple Approach for Enabling Federated Learning on Resource Constrained Devices

arXiv: https://arxiv.org/abs/2109.15258 · HTML: https://mlsanity.com/paper/2109.15258/

## Key facts

- Cited by 56 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 21 references that are in the map.
- First posted on arXiv 30 Sep 2021.

## Builds on (most cited first, 11 of 11)

- [A Joint Learning and Communications Framework for Federated Learning over Wireless Networks](https://mlsanity.com/paper/1909.07972/) (2019)
- [Federated Learning via Over-the-Air Computation](https://mlsanity.com/paper/1812.11750/) (2018)
- [Broadband Analog Aggregation for Low-Latency Federated Edge Learning (Extended Version)](https://mlsanity.com/paper/1812.11494/) (2018)
- [The Roadmap to 6G -- AI Empowered Wireless Networks](https://mlsanity.com/paper/1904.11686/) (2019)
- [Scheduling Policies for Federated Learning in Wireless Networks](https://mlsanity.com/paper/1908.06287/) (2019)
- [Dropout Training as Adaptive Regularization](https://mlsanity.com/paper/1307.1493/) (2013)
- [Scheduling for Cellular Federated Edge Learning with Importance and Channel Awareness](https://mlsanity.com/paper/2004.00490/) (2020)
- [Device Scheduling with Fast Convergence for Wireless Federated Learning](https://mlsanity.com/paper/1911.00856/) (2019)
- [High-Dimensional Stochastic Gradient Quantization for Communication-Efficient Edge Learning](https://mlsanity.com/paper/1910.03865/) (2019)
- [Joint Parameter-and-Bandwidth Allocation for Improving the Efficiency of Partitioned Edge Learning](https://mlsanity.com/paper/2003.04544/) (2020)
- [Adaptive Subcarrier, Parameter, and Power Allocation for Partitioned Edge Learning Over Broadband Channels](https://mlsanity.com/paper/2010.04061/) (2020)

## Built on by (most cited first, 30 of 56)

- [Task-Oriented Sensing, Computation, and Communication Integration for Multi-Device Edge AI](https://mlsanity.com/paper/2207.00969/) (2022)
- [Streaming DiLoCo with overlapping communication: Towards a Distributed Free Lunch](https://mlsanity.com/paper/2501.18512/) (2025)
- [Green Edge AI: A Contemporary Survey](https://mlsanity.com/paper/2312.00333/) (2023)
- [FedOBD: Opportunistic Block Dropout for Efficiently Training Large-scale Neural Networks through Federated Learning](https://mlsanity.com/paper/2208.05174/) (2022)
- [An Upload-Efficient Scheme for Transferring Knowledge From a Server-Side Pre-trained Generator to Clients in Heterogeneous Federated Learning](https://mlsanity.com/paper/2403.15760/) (2024)
- [Task-Oriented Over-the-Air Computation for Multi-Device Edge AI](https://mlsanity.com/paper/2211.01255/) (2022)
- [Complement Sparsification: Low-Overhead Model Pruning for Federated Learning](https://mlsanity.com/paper/2303.06237/) (2023)
- [A Survey on Integrated Sensing, Communication, and Computation](https://mlsanity.com/paper/2408.08074/) (2024)
- [Convergence Acceleration in Wireless Federated Learning: A Stackelberg Game Approach](https://mlsanity.com/paper/2209.06623/) (2022)
- [Adaptive Federated Pruning in Hierarchical Wireless Networks](https://mlsanity.com/paper/2305.09042/) (2023)
- [FLrce: Resource-Efficient Federated Learning with Early-Stopping Strategy](https://mlsanity.com/paper/2310.09789/) (2023)
- [Spectrum Breathing: Protecting Over-the-Air Federated Learning Against Interference](https://mlsanity.com/paper/2305.05933/) (2023)
- [WASH: Train your Ensemble with Communication-Efficient Weight Shuffling, then Average](https://mlsanity.com/paper/2405.17517/) (2024)
- [Balancing Accuracy and Integrity for Reconfigurable Intelligent Surface-aided Over-the-Air Federated Learning](https://mlsanity.com/paper/2207.08057/) (2022)
- [Federated Learning While Providing Model as a Service: Joint Training and Inference Optimization](https://mlsanity.com/paper/2312.12863/) (2023)
- [FedLP: Layer-wise Pruning Mechanism for Communication-Computation Efficient Federated Learning](https://mlsanity.com/paper/2303.06360/) (2023)
- [Leveraging Randomness in Model and Data Partitioning for Privacy Amplification](https://mlsanity.com/paper/2503.03043/) (2025)
- [Towards a Better Theoretical Understanding of Independent Subnetwork Training](https://mlsanity.com/paper/2306.16484/) (2023)
- [Federated Neural Radiance Fields](https://mlsanity.com/paper/2305.01163/) (2023)
- [FedBIAD: Communication-Efficient and Accuracy-Guaranteed Federated Learning with Bayesian Inference-Based Adaptive Dropout](https://mlsanity.com/paper/2307.07172/) (2023)
- [Exploring the Practicality of Federated Learning: A Survey Towards the Communication Perspective](https://mlsanity.com/paper/2405.20431/) (2024)
- [F3OCUS -- Federated Finetuning of Vision-Language Foundation Models with Optimal Client Layer Updating Strategy via Multi-objective Meta-Heuristics](https://mlsanity.com/paper/2411.11912/) (2024)
- [Training Latency Minimization for Model-Splitting Allowed Federated Edge Learning](https://mlsanity.com/paper/2307.11532/) (2023)
- [Equitable Federated Learning with NCA](https://mlsanity.com/paper/2506.21735/) (2025)
- [Model Parallelism With Subnetwork Data Parallelism](https://mlsanity.com/paper/2507.09029/) (2025)
- [Sparse Decentralized Federated Learning](https://mlsanity.com/paper/2308.16671/) (2023)
- [FedDD: Toward Communication-efficient Federated Learning with Differential Parameter Dropout](https://mlsanity.com/paper/2308.16835/) (2023)
- [Device Sampling and Resource Optimization for Federated Learning in Cooperative Edge Networks](https://mlsanity.com/paper/2311.04350/) (2023)
- [Straggler-resilient Federated Learning: Tackling Computation Heterogeneity with Layer-wise Partial Model Training in Mobile Edge Network](https://mlsanity.com/paper/2311.10002/) (2023)
- [Energy-Efficient Edge Learning via Joint Data Deepening-and-Prefetching](https://mlsanity.com/paper/2402.11925/) (2024)

## Abstract (arXiv)

Federated learning (FL) is a popular framework for training an AI model using distributed mobile data in a wireless network. It features data parallelism by distributing the learning task to multiple edge devices while attempting to preserve their local-data privacy. One main challenge confronting practical FL is that resource constrained devices struggle with the computation intensive task of updating of a deep-neural network model. To tackle the challenge, in this paper, a federated dropout (FedDrop) scheme is proposed building on the classic dropout scheme for random model pruning. Specifically, in each iteration of the FL algorithm, several subnets are independently generated from the global model at the server using dropout but with heterogeneous dropout rates (i.e., parameter-pruning probabilities),each of which is adapted to the state of an assigned channel. The subnets are downloaded to associated devices for updating. Thereby, FedDrop reduces both the communication overhead and devices' computation loads compared with the conventional FL while outperforming the latter in the case of overfitting and also the FL scheme with uniform dropout (i.e., identical subnets).