mlsanityData to 13 Oct 2026

Federated Learning for Computationally-Constrained Heterogeneous Devices: A Survey

arXiv 2307.09182 ↗First posted 18 Jul 2023cs.LGcited by 40

Not on the map. Why?

Abstract (arXiv)

With an increasing number of smart devices like internet of things (IoT) devices deployed in the field, offloadingtraining of neural networks (NNs) to a central server becomes more and more infeasible. Recent efforts toimprove users' privacy have led to on-device learning emerging as an alternative. However, a model trainedonly on a single device, using only local data, is unlikely to reach a high accuracy. Federated learning (FL)has been introduced as a solution, offering a privacy-preserving trade-off between communication overheadand model accuracy by sharing knowledge between devices but disclosing the devices' private data. Theapplicability and the benefit of applying baseline FL are, however, limited in many relevant use cases dueto the heterogeneity present in such environments. In this survey, we outline the heterogeneity challengesFL has to overcome to be widely applicable in real-world applications. We especially focus on the aspect ofcomputation heterogeneity among the participating devices and provide a comprehensive overview of recentworks on heterogeneity-aware FL. We discuss two groups: works that adapt the NN architecture and worksthat approach heterogeneity on a system level, covering Federated Averaging (FedAvg), distillation, and splitlearning-based approaches, as well as synchronous and asynchronous aggregation schemes.

Key facts. Cited by 40 papers in the map (data through 13 Oct 2026). Builds on 73 of its 115 references that are in the map. First posted on arXiv 18 Jul 2023.

Builds on

73 of 115 references.

References · 73

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,950
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,223
  3. 2015Distilling the Knowledge in a Neural Networkcited by 10,180
  4. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,653
  5. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,426
  6. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,836
  7. 2019Advances and Open Problems in Federated Learningcited by 3,040
  8. 2018Federated Optimization in Heterogeneous Networkscited by 2,981
  9. 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,538
  10. 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,470
  11. 2017ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devicescited by 1,199
  12. 2019Towards Federated Learning at Scale: System Designcited by 830
38 more
  1. 2018LEAF: A Benchmark for Federated Settingscited by 789
  2. 2020Ensemble Distillation for Robust Model Fusion in Federated Learningcited by 682
  3. 2018Local SGD Converges Fast and Communicates Littlecited by 601
  4. 2020Federated Learning with Matched Averagingcited by 564
  5. 2019Federated Learning with Differential Privacy: Algorithms and Performance Analysiscited by 536
  6. 2019FedMD: Heterogenous Federated Learning via Model Distillationcited by 459
  7. 2018Adaptive Federated Learning in Resource Constrained Edge Computing Systemscited by 409
  8. 2019Bayesian Nonparametric Federated Learning of Neural Networkscited by 387
  9. 2019FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantizationcited by 356
  10. 2018Client Selection for Federated Learning with Heterogeneous Resources in Mobile Edgecited by 350
  11. 2020HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clientscited by 337
  12. 2019Asynchronous Federated Optimizationcited by 265
  13. 2019Model Pruning Enables Efficient Federated Learning on Edge Devicescited by 221
  14. 2017EMNIST: an extension of MNIST to handwritten letterscited by 213
  15. 2018Applied Federated Learning: Improving Google Keyboard Query Suggestionscited by 204
  16. 2021FjORD: Fair and Accurate Federated Learning under heterogeneous targets with Ordered Dropoutcited by 192
  17. 2019FedHealth: A Federated Transfer Learning Framework for Wearable Healthcarecited by 182
  18. 2019Federated Learning over Wireless Fading Channelscited by 182
  19. 2016Practical Secure Aggregation for Federated Learning on User-Held Datacited by 178
  20. 2018Expanding the Reach of Federated Learning by Reducing Client Resource Requirementscited by 162
  21. 2020Survey of Personalization Techniques for Federated Learningcited by 144
  22. 2017Revisiting Distributed Synchronous SGDcited by 142
  23. 2020Federated Learning for Internet of Things: Recent Advances, Taxonomy, and Open Challengescited by 139
  24. 2022FedRolex: Model-Heterogeneous Federated Learning with Rolling Sub-Model Extractioncited by 102
  25. 2020TiFL: A Tier-based Federated Learning Systemcited by 96
  26. 2020Communication-Efficient Edge AI: Algorithms and Systemscited by 95
  27. 2019Cronus: Robust and Heterogeneous Collaborative Learning with Black-Box Knowledge Transfercited by 93
  28. 2021Asynchronous Federated Learning on Heterogeneous Devices: A Surveycited by 93
  29. 2021Papaya: Practical, Private, and Scalable Federated Learningcited by 90
  30. 2019Lifelong Federated Reinforcement Learning: A Learning Architecture for Navigation in Cloud Robotic Systemscited by 77
  31. 2020Federated Mutual Learningcited by 71
  32. 2018NestDNN: Resource-Aware Multi-Tenant On-Device Deep Learning for Continuous Mobile Visioncited by 68
  33. 2020FedAT: A High-Performance and Communication-Efficient Federated Learning System with Asynchronous Tierscited by 64
  34. 2019On-Device Machine Learning: An Algorithms and Learning Theory Perspectivecited by 63
  35. 2020Straggler-Resilient Federated Learning: Leveraging the Interplay Between Statistical Accuracy and System Heterogeneitycited by 56
  36. 2019Energy Demand Prediction with Federated Learning for Electric Vehicle Networkscited by 46
  37. 2020Dynamic Tensor Rematerializationcited by 45
  38. 2022ZeroFL: Efficient On-Device Training for Federated Learning with Local Sparsitycited by 45

50 most cited of 73 All 73 (JSON)

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2024: 18; 2025: 10; 2026: 12.

40 papers.

Sort:

By area:

  1. 2024The Future of Large Language Model Pre-training is Federatedcited by 30
  2. 2024Vertical Federated Learning for Effectiveness, Security, Applicability: A Surveycited by 14
  3. 2024Personalized Federated Learning via Feature Distribution Adaptationcited by 13
  4. 2024Advances in Robust Federated Learning: A Survey with Heterogeneity Considerationscited by 12
  5. 2024Green Federated Learning: A new era of Green Aware AIcited by 12
  6. 2024Towards Diverse Device Heterogeneous Federated Learning via Task Arithmetic Knowledge Integrationcited by 5
  7. 2025Feature Distillation is the Better Choice for Model-Heterogeneous Federated Learningcited by 5
  8. 2024Stragglers-Aware Low-Latency Synchronous Federated Learning via Layer-Wise Model Updatescited by 4
  9. 2024Exploring the Practicality of Federated Learning: A Survey Towards the Communication Perspectivecited by 4
  10. 2024SignSGD with Federated Votingcited by 2
  11. 2024Using Diffusion Models as Generative Replay in Continual Federated Learning -- What will Happen?cited by 2
  12. 2025A survey on Clustered Federated Learning: Taxonomy, Analysis and Applicationscited by 2
28 more
  1. 2025FedHQ: Hybrid Runtime Quantization for Federated Learningcited by 2
  2. 2026Collaborative Adaptive Curriculum for Progressive Knowledge Distillationcited by 2
  3. 2024Federated Learning with Flexible Architecturescited by 1
  4. 2024Efficient Federated Finetuning of Tiny Transformers with Resource-Constrained Devicescited by 1
  5. 2024Federated Learning with Uncertainty and Personalization via Efficient Second-order Optimizationcited by 1
  6. 2025Aggregation on Learnable Manifolds for Asynchronous Federated Optimizationcited by 1
  7. 2025Adaptive Deadline and Batch Layered Synchronized Federated Learningcited by 1
  8. 2026CRAD: Class-wise Reliability-Aware Distillation for Decentralized Heterogeneous Federated Learningcited by 1
  9. 2024Federated Computing -- Survey on Building Blocks, Extensions and Systemscited by 0
  10. 2024FedMap: Iterative Magnitude-Based Pruning for Communication-Efficient Federated Learningcited by 0
  11. 2024On Sampling Strategies for Spectral Model Shardingcited by 0
  12. 2024Predicting Quality of Video Gaming Experience Using Global-Scale Telemetry Data and Federated Learningcited by 0
  13. 2024GeFL: Model-Agnostic Federated Learning with Generative Modelscited by 0
  14. 2025Efficient Zeroth-Order Federated Finetuning of Language Models on Resource-Constrained Devicescited by 0
  15. 2025Federated Smoothing ADMM for Localizationcited by 0
  16. 2025EMO: Edge Model Overlays to Scale Model Size in Federated Learningcited by 0
  17. 2025Federated Spectral Graph Transformers Meet Neural Ordinary Differential Equations for Non-IID Graphscited by 0
  18. 2025CEPerFed: Communication-Efficient Personalized Federated Learning for Multi-Pulse MRI Classificationcited by 0
  19. 2026RIFLE: Robust Distillation-based FL for Deep Model Deployment on Resource-Constrained IoT Networkscited by 0
  20. 2026Personalized Federated Learning via Gaussian Generative Modelingcited by 0
  21. 2026Agentic Federated Learning: The Future of Distributed Training Orchestrationcited by 0
  22. 2026VARS-FL: Validation-Aligned Client Selection for Non-IID Federated Learning in IoT Systemscited by 0
  23. 2026UB-SMoE: Universally Balanced Sparse Mixture-of-Experts for Resource-adaptive Federated Fine-tuning of Foundation Modelscited by 0
  24. 2026Parameter Efficient Multi-Class Intelligent Scheduling for Multimodal Online Distributed Industrial Anomaly Detectioncited by 0
  25. 2026Hardware-Aware Federated Learning for Speech Emotion Recognitioncited by 0
  26. 2026Quantization in Federated Learning: Methods, Challenges and Future Directionscited by 0
  27. 2026A Game-Theoretic Framework for Incentive-Compatible AI training Under Renewable-Energy Constraintscited by 0
  28. 2026FedFIbOS: Fisher Importance based Optimal Submodelling for Heterogeneous Federated Learningcited by 0