mlsanityData to 7 Oct 2026

Federated Dropout -- A Simple Approach for Enabling Federated Learning on Resource Constrained Devices

arXiv 2109.15258 ↗First posted 30 Sep 2021cs.LG cs.IT math.ITcited by 56

Not on the map. Why?

Abstract (arXiv)

Federated learning (FL) is a popular framework for training an AI model using distributed mobile data in a wireless network. It features data parallelism by distributing the learning task to multiple edge devices while attempting to preserve their local-data privacy. One main challenge confronting practical FL is that resource constrained devices struggle with the computation intensive task of updating of a deep-neural network model. To tackle the challenge, in this paper, a federated dropout (FedDrop) scheme is proposed building on the classic dropout scheme for random model pruning. Specifically, in each iteration of the FL algorithm, several subnets are independently generated from the global model at the server using dropout but with heterogeneous dropout rates (i.e., parameter-pruning probabilities),each of which is adapted to the state of an assigned channel. The subnets are downloaded to associated devices for updating. Thereby, FedDrop reduces both the communication overhead and devices' computation loads compared with the conventional FL while outperforming the latter in the case of overfitting and also the FL scheme with uniform dropout (i.e., identical subnets).

Key facts. Cited by 56 papers in the map (data through 7 Oct 2026). Builds on 11 of its 21 references that are in the map. First posted on arXiv 30 Sep 2021.

Builds on

11 of 21 references.

References · 11

  1. 2019A Joint Learning and Communications Framework for Federated Learning over Wireless Networkscited by 262
  2. 2018Federated Learning via Over-the-Air Computationcited by 215
  3. 2018Broadband Analog Aggregation for Low-Latency Federated Edge Learning (Extended Version)cited by 186
  4. 2019The Roadmap to 6G -- AI Empowered Wireless Networkscited by 150
  5. 2019Scheduling Policies for Federated Learning in Wireless Networkscited by 131
  6. 2013Dropout Training as Adaptive Regularizationcited by 89
  7. 2020Scheduling for Cellular Federated Edge Learning with Importance and Channel Awarenesscited by 62
  8. 2019Device Scheduling with Fast Convergence for Wireless Federated Learningcited by 46
  9. 2019High-Dimensional Stochastic Gradient Quantization for Communication-Efficient Edge Learningcited by 32
  10. 2020Joint Parameter-and-Bandwidth Allocation for Improving the Efficiency of Partitioned Edge Learningcited by 15
  11. 2020Adaptive Subcarrier, Parameter, and Power Allocation for Partitioned Edge Learning Over Broadband Channelscited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 6; 2023: 18; 2024: 13; 2025: 14; 2026: 5.

56 papers.

Sort:

By area:

  1. 2022Task-Oriented Sensing, Computation, and Communication Integration for Multi-Device Edge AIcited by 31
  2. 2025Streaming DiLoCo with overlapping communication: Towards a Distributed Free Lunchcited by 26
  3. 2023Green Edge AI: A Contemporary Surveycited by 24
  4. 2022FedOBD: Opportunistic Block Dropout for Efficiently Training Large-scale Neural Networks through Federated Learningcited by 17
  5. 2024An Upload-Efficient Scheme for Transferring Knowledge From a Server-Side Pre-trained Generator to Clients in Heterogeneous Federated Learningcited by 17
  6. 2022Task-Oriented Over-the-Air Computation for Multi-Device Edge AIcited by 12
  7. 2023Complement Sparsification: Low-Overhead Model Pruning for Federated Learningcited by 12
  8. 2024A Survey on Integrated Sensing, Communication, and Computationcited by 12
  9. 2022Convergence Acceleration in Wireless Federated Learning: A Stackelberg Game Approachcited by 10
  10. 2023Adaptive Federated Pruning in Hierarchical Wireless Networkscited by 10
  11. 2023FLrce: Resource-Efficient Federated Learning with Early-Stopping Strategycited by 10
  12. 2023Spectrum Breathing: Protecting Over-the-Air Federated Learning Against Interferencecited by 9
38 more
  1. 2024WASH: Train your Ensemble with Communication-Efficient Weight Shuffling, then Averagecited by 9
  2. 2023Federated Learning While Providing Model as a Service: Joint Training and Inference Optimizationcited by 8
  3. 2022Balancing Accuracy and Integrity for Reconfigurable Intelligent Surface-aided Over-the-Air Federated Learningcited by 7
  4. 2023FedLP: Layer-wise Pruning Mechanism for Communication-Computation Efficient Federated Learningcited by 7
  5. 2025Leveraging Randomness in Model and Data Partitioning for Privacy Amplificationcited by 7
  6. 2023Towards a Better Theoretical Understanding of Independent Subnetwork Trainingcited by 6
  7. 2023Federated Neural Radiance Fieldscited by 4
  8. 2023FedBIAD: Communication-Efficient and Accuracy-Guaranteed Federated Learning with Bayesian Inference-Based Adaptive Dropoutcited by 4
  9. 2024Exploring the Practicality of Federated Learning: A Survey Towards the Communication Perspectivecited by 4
  10. 2024F3OCUS -- Federated Finetuning of Vision-Language Foundation Models with Optimal Client Layer Updating Strategy via Multi-objective Meta-Heuristicscited by 4
  11. 2023Training Latency Minimization for Model-Splitting Allowed Federated Edge Learningcited by 3
  12. 2025Equitable Federated Learning with NCAcited by 3
  13. 2025Model Parallelism With Subnetwork Data Parallelismcited by 3
  14. 2023Sparse Decentralized Federated Learningcited by 2
  15. 2023FedDD: Toward Communication-efficient Federated Learning with Differential Parameter Dropoutcited by 2
  16. 2023Device Sampling and Resource Optimization for Federated Learning in Cooperative Edge Networkscited by 2
  17. 2023Straggler-resilient Federated Learning: Tackling Computation Heterogeneity with Layer-wise Partial Model Training in Mobile Edge Networkcited by 2
  18. 2024Energy-Efficient Edge Learning via Joint Data Deepening-and-Prefetchingcited by 2
  19. 2024Learnable Sparse Customization in Heterogeneous Edge Computingcited by 2
  20. 2024Federated Domain Generalization with Label Smoothing and Balanced Decentralized Trainingcited by 2
  21. 2025Efficient Federated Learning with Heterogeneous Data and Adaptive Dropoutcited by 2
  22. 2025FedLoDrop: Federated LoRA with Dropout for Generalized LLM Fine-tuningcited by 2
  23. 2023Digital Over-the-Air Federated Learning in Multi-Antenna Systemscited by 1
  24. 2025FedORGP: Guiding Heterogeneous Federated Learning with Orthogonality Regularization on Global Prototypescited by 1
  25. 2025Pareto Actor-Critic for Communication and Computation Co-Optimization in Non-Cooperative Federated Learning Servicescited by 1
  26. 2025Federated Learning via Meta-Variational Dropoutcited by 1
  27. 2025TT-Prune: Joint Model Pruning and Resource Allocation for Communication-efficient Time-triggered Federated Learningcited by 1
  28. 2022Fast Server Learning Rate Tuning for Coded Federated Dropoutcited by 0
  29. 2023CRS-FL: Conditional Random Sampling for Communication-Efficient and Privacy-Preserving Federated Learningcited by 0
  30. 2023VREM-FL: Mobility-Aware Computation-Scheduling Co-Design for Vehicular Federated Learningcited by 0
  31. 2024WHALE-FL: Wireless and Heterogeneity Aware Latency Efficient Federated Learning over Mobile Devices via Adaptive Subnetwork Schedulingcited by 0
  32. 2024Selective Knowledge Sharing for Personalized Federated Learning Under Capacity Heterogeneitycited by 0
  33. 2024FedProphet: Memory-Efficient Federated Adversarial Training via Robust and Consistent Cascade Learningcited by 0
  34. 2024Adaptive Guidance for Local Training in Heterogeneous Federated Learningcited by 0
  35. 2024Hybrid-Regularized Magnitude Pruning for Robust Federated Learning under Covariate Shiftcited by 0
  36. 2025HEART: Achieving Timely Multi-Model Training for Vehicle-Edge-Cloud-Integrated Hierarchical Federated Learningcited by 0
  37. 2025Traversal Learning: A Lossless And Efficient Distributed Learning Frameworkcited by 0
  38. 2025Addressing the Collaboration Dilemma in Low-Data Federated Learning via Transient Sparsitycited by 0

50 most cited of 56 All 56 (JSON)