mlsanityData to 7 Oct 2026

Training Machine Learning models at the Edge: A Survey

arXiv 2403.02619 ↗First posted 5 Mar 2024cs.LG cs.DCcited by 11

Not on the map. Why?

Abstract (arXiv)

Edge computing has gained significant traction in recent years, promising enhanced efficiency by integrating artificial intelligence capabilities at the edge. While the focus has primarily been on the deployment and inference of Machine Learning (ML) models at the edge, the training aspect remains less explored. This survey, explores the concept of edge learning, specifically the optimization of ML model training at the edge. The objective is to comprehensively explore diverse approaches and methodologies in edge learning, synthesize existing knowledge, identify challenges, and highlight future trends. Utilizing Scopus and Web of science advanced search, relevant literature on edge learning was identified, revealing a concentration of research efforts in distributed learning methods, particularly federated learning. This survey further provides a guideline for comparing techniques used to optimize ML for edge learning, along with an exploration of the different frameworks, libraries, and simulation tools available. In doing so, the paper contributes to a holistic understanding of the current landscape and future directions in the intersection of edge computing and machine learning, paving the way for informed comparisons between optimization methods and techniques designed for training on the edge.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 176 of its 443 references that are in the map. First posted on arXiv 5 Mar 2024.

Builds on

176 of 443 references.

References · 176

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  5. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  6. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  7. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  8. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,286
  9. 2024Mixtral of Expertscited by 1,785
  10. 2020Knowledge Distillation: A Surveycited by 1,578
  11. 2019On the Convergence of FedAvg on Non-IID Datacited by 1,220
  12. 2019A Comprehensive Survey on Transfer Learningcited by 1,154
38 more
  1. 2018On First-Order Meta-Learning Algorithmscited by 1,003
  2. 2020Meta-Learning in Neural Networks: A Surveycited by 944
  3. 2021A Survey of Quantization Methods for Efficient Neural Network Inferencecited by 798
  4. 2018LEAF: A Benchmark for Federated Settingscited by 780
  5. 2019Large Scale Incremental Learningcited by 727
  6. 2016TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systemscited by 660
  7. 2019Federated Learning with Differential Privacy: Algorithms and Performance Analysiscited by 528
  8. 2019Quantifying the Carbon Emissions of Machine Learningcited by 516
  9. 2019Clustered Federated Learning: Model-Agnostic Distributed Multi-Task Optimization under Privacy Constraintscited by 499
  10. 2020Flower: A Friendly Federated Learning Research Frameworkcited by 449
  11. 2017Safe Reinforcement Learning via Shieldingcited by 427
  12. 2018A Survey on Deep Transfer Learningcited by 417
  13. 2023Scalable Extraction of Training Data from (Production) Language Modelscited by 415
  14. 2019Federated Learning in Mobile Edge Networks: A Comprehensive Surveycited by 407
  15. 2021Federated Learning on Non-IID Data: A Surveycited by 345
  16. 2021FNet: Mixing Tokens with Fourier Transformscited by 343
  17. 2023AudioLDM 2: Learning Holistic Audio Generation with Self-supervised Pretrainingcited by 343
  18. 2020SplitFed: When Federated Learning Meets Split Learningcited by 308
  19. 2020FedML: A Research Library and Benchmark for Federated Machine Learningcited by 298
  20. 2022The Forward-Forward Algorithm: Some Preliminary Investigationscited by 262
  21. 2023A Cookbook of Self-Supervised Learningcited by 248
  22. 2019Model Pruning Enables Efficient Federated Learning on Edge Devicescited by 219
  23. 2019Client-Edge-Cloud Hierarchical Federated Learningcited by 202
  24. 2019Low-bit Quantization of Neural Networks for Efficient Inferencecited by 185
  25. 2022A Fast Post-Training Pruning Framework for Transformerscited by 161
  26. 2022On-Device Training Under 256KB Memorycited by 158
  27. 2023Text-to-Audio Generation using Instruction-Tuned LLM and Latent Diffusion Modelcited by 145
  28. 2020Federated Visual Classification with Real-World Data Distributioncited by 126
  29. 2019Convergence of Edge Computing and Deep Learning: A Comprehensive Surveycited by 120
  30. 2021Overcoming Catastrophic Forgetting in Incremental Few-Shot Learning by Finding Flat Minimacited by 118
  31. 2021Data Collection and Quality Challenges in Deep Learning: A Data-Centric AI Perspectivecited by 116
  32. 2019Sequential Recommender Systems: Challenges, Progress and Prospectscited by 115
  33. 2020One-Bit Over-the-Air Aggregation for Communication-Efficient Federated Edge Learning: Design and Convergence Analysiscited by 108
  34. 2020A Survey of Deep Meta-Learningcited by 99
  35. 2019Hierarchical Federated Learning Across Heterogeneous Cellular Networkscited by 98
  36. 2020Communication-Efficient Edge AI: Algorithms and Systemscited by 95
  37. 2021Heterogeneity for the Win: One-Shot Federated Clusteringcited by 87
  38. 2023Personalisation within bounds: A risk taxonomy and policy framework for the alignment of large language models with personalised feedbackcited by 86

50 most cited of 176 All 176 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 5; 2026: 3.

11 papers.

  1. 2024On-Device Language Models: A Comprehensive Reviewcited by 101
  2. 2024Self-Contrastive Forward-Forward Algorithmcited by 15
  3. 2025Training-Free Dataset Pruning for Instance Segmentationcited by 7
  4. 2025Tin-Tin: Towards Tiny Learning on Tiny Devices with Integer-based Neural Network Trainingcited by 1
  5. 2025Decentor-V: Lightweight ML Training on Low-Power RISC-V Edge Devicescited by 1
  6. 2024Data Quality in Edge Machine Learning: A State-of-the-Art Surveycited by 0
  7. 2025Vision-Based Embedded System for Noncontact Monitoring of Preterm Infant Behavior in Low-Resource Care Settingscited by 0
  8. 2025Online Continual Learning on Intel Loihi 2 via a Co-designed Spiking Neural Networkcited by 0
  9. 2026Uncertainty-Guided Edge Learning for Deep Image Regression in Remote Sensingcited by 0
  10. 2026Thermo-FL: Thermal-Aware Robust Federated Fine-Tuning of Large Language Models for Edge AIcited by 0
  11. 2026Towards Stream Learning on Embedded Systems: Benchmarking the Memory Consumption of Stream Learning Methodscited by 0