mlsanityData to 14 Oct 2026

Efficient Federated Learning with Heterogeneous Data and Adaptive Dropout

arXiv 2507.10430 ↗First posted 14 Jul 2025cs.DC cs.AI cs.LGcited by 2

Not on the map. Why?

Abstract (arXiv)

Federated Learning (FL) is a promising distributed machine learning approach that enables collaborative training of a global model using multiple edge devices. The data distributed among the edge devices is highly heterogeneous. Thus, FL faces the challenge of data distribution and heterogeneity, where non-Independent and Identically Distributed (non-IID) data across edge devices may yield in significant accuracy drop. Furthermore, the limited computation and communication capabilities of edge devices increase the likelihood of stragglers, thus leading to slow model convergence. In this paper, we propose the FedDHAD FL framework, which comes with two novel methods: Dynamic Heterogeneous model aggregation (FedDH) and Adaptive Dropout (FedAD). FedDH dynamically adjusts the weights of each local model within the model aggregation process based on the non-IID degree of heterogeneous data to deal with the statistical data heterogeneity. FedAD performs neuron-adaptive operations in response to heterogeneous devices to improve accuracy while achieving superb efficiency. The combination of these two methods makes FedDHAD significantly outperform state-of-the-art solutions in terms of accuracy (up to 6.7% higher), efficiency (up to 2.02 times faster), and computation cost (up to 15.0% smaller).

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 49 of its 76 references that are in the map. First posted on arXiv 14 Jul 2025.

Builds on

49 of 76 references.

References · 49

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  3. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  4. 2019Advances and Open Problems in Federated Learningcited by 3,046
  5. 2018Federated Optimization in Heterogeneous Networkscited by 2,987
  6. 2019On the Convergence of FedAvg on Non-IID Datacited by 1,229
  7. 2020Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimizationcited by 906
  8. 2019Towards Federated Learning at Scale: System Designcited by 832
  9. 2021Model-Contrastive Federated Learningcited by 766
  10. 2020Ensemble Distillation for Robust Model Fusion in Federated Learningcited by 682
  11. 2017Federated Multi-Task Learningcited by 672
  12. 2021Federated Learning on Non-IID Data Silos: An Experimental Studycited by 568
37 more
  1. 2021Federated Learning Based on Dynamic Regularizationcited by 562
  2. 2019Bayesian Nonparametric Federated Learning of Neural Networkscited by 387
  3. 2020HRank: Filter Pruning using High-Rank Feature Mapcited by 292
  4. 2018Approximation Methods for Bilevel Programmingcited by 236
  5. 2019Model Pruning Enables Efficient Federated Learning on Edge Devicescited by 222
  6. 2021FjORD: Fair and Accurate Federated Learning under heterogeneous targets with Ordered Dropoutcited by 192
  7. 2019Adaptive Gradient-Based Meta-Learning Methodscited by 183
  8. 2022Federated Learning from Pre-Trained Models: A Contrastive Learning Approachcited by 142
  9. 2021Multi-Center Federated Learning: Clients Clustering for Better Personalizationcited by 113
  10. 2021From Distributed Machine Learning to Federated Learning: A Surveycited by 66
  11. 2023Federated Learning of Large Language Models with Parameter-Efficient Prompt Tuning and Adaptive Optimizationcited by 65
  12. 2019Astraea: Self-balancing Federated Learning for Improving Classification Accuracy of Mobile Deep Learning Applicationscited by 64
  13. 2021Federated Dropout -- A Simple Approach for Enabling Federated Learning on Resource Constrained Devicescited by 56
  14. 2021Federated Dynamic Sparse Training: Computing Less, Communicating Less, Yet Learning Bettercited by 55
  15. 2019Semi-Cyclic Stochastic Gradient Descentcited by 40
  16. 2023Generalizable Heterogeneous Federated Cross-Correlation and Instance Similarity Learningcited by 40
  17. 2020Adaptive Federated Dropout: Improving Communication Efficiency and Generalization for Federated Learningcited by 38
  18. 2023Exploring One-shot Semi-supervised Federated Learning with A Pre-trained Diffusion Modelcited by 33
  19. 2023FedASMU: Efficient Asynchronous Federated Learning with Dynamic Staleness-aware Model Updatecited by 32
  20. 2024Fair Federated Learning under Domain Skew with Local Consistency and Domain Diversitycited by 30
  21. 2021Efficient Device Scheduling with Multi-Job Federated Learningcited by 29
  22. 2022FedDUAP: Federated Learning with Dynamic Update and Adaptive Pruning Using Shared Data on the Servercited by 29
  23. 2023Grounding Foundation Models through Federated Transfer Learning: A General Frameworkcited by 24
  24. 2019One-Shot Federated Learning: Theoretical Limits and Algorithms to Achieve Themcited by 23
  25. 2022Accelerated Federated Learning with Decoupled Adaptive Optimizationcited by 23
  26. 2022Multi-Job Intelligent Scheduling with Cross-Device Federated Learningcited by 20
  27. 2024An Upload-Efficient Scheme for Transferring Knowledge From a Server-Side Pre-trained Generator to Clients in Heterogeneous Federated Learningcited by 17
  28. 2021HeterPS: Distributed Deep Learning With Reinforcement Learning Based Scheduling in Heterogeneous Environmentscited by 15
  29. 2023AEDFL: Efficient Asynchronous Decentralized Federated Learning with Heterogeneous Devicescited by 15
  30. 2024Trustworthy Federated Learning: Privacy, Security, and Beyondcited by 11
  31. 2023Efficient Asynchronous Federated Learning with Sparsification and Quantizationcited by 9
  32. 2024Enhancing Trust and Privacy in Distributed Networks: A Comprehensive Survey on Blockchain-based Federated Learningcited by 9
  33. 2024Efficient Federated Learning Using Dynamic Update and Adaptive Pruning with Momentum on Shared Server Datacited by 6
  34. 2024Fisher Information-based Efficient Curriculum Federated Learning with Large Language Modelscited by 6
  35. 2025AugFL: Augmenting Federated Learning with Pretrained Modelscited by 6
  36. 2023Benchmarking FedAvg and FedCurv for Image Classification Taskscited by 3
  37. 2025Efficient Federated Learning with Timely Update Disseminationcited by 2

Built on by

Cited by year

Citing papers per year: 2025: 2.

2 papers.

  1. 2025Bridging Symmetry and Robustness: On the Role of Equivariance in Enhancing Adversarial Robustnesscited by 8
  2. 2025LENS: LLM-Enabled Narrative Synthesis for Mental Health by Aligning Multimodal Sensing with Language Modelscited by 3