mlsanityData to 14 Oct 2026

Parallel Successive Learning for Dynamic Distributed Model Training over Heterogeneous Wireless Networks

arXiv 2202.02947 ↗First posted 7 Feb 2022cs.LG cs.AI cs.NIcited by 18

Not on the map. Why?

Abstract (arXiv)

Federated learning (FedL) has emerged as a popular technique for distributing model training over a set of wireless devices, via iterative local updates (at devices) and global aggregations (at the server). In this paper, we develop parallel successive learning (PSL), which expands the FedL architecture along three dimensions: (i) Network, allowing decentralized cooperation among the devices via device-to-device (D2D) communications. (ii) Heterogeneity, interpreted at three levels: (ii-a) Learning: PSL considers heterogeneous number of stochastic gradient descent iterations with different mini-batch sizes at the devices; (ii-b) Data: PSL presumes a dynamic environment with data arrival and departure, where the distributions of local datasets evolve over time, captured via a new metric for model/concept drift. (ii-c) Device: PSL considers devices with different computation and communication capabilities. (iii) Proximity, where devices have different distances to each other and the access point. PSL considers the realistic scenario where global aggregations are conducted with idle times in-between them for resource efficiency improvements, and incorporates data dispersion and model dispersion with local model condensation into FedL. Our analysis sheds light on the notion of cold vs. warmed up models, and model inertia in distributed machine learning. We then propose network-aware dynamic model tracking to optimize the model learning vs. resource efficiency tradeoff, which we show is an NP-hard signomial programming problem. We finally solve this problem through proposing a general optimization solver. Our numerical results reveal new findings on the interdependencies between the idle times in-between the global aggregations, model/concept drift, and D2D cooperation configuration.

Key facts. Cited by 18 papers in the map (data through 14 Oct 2026). Builds on 31 of its 61 references that are in the map. First posted on arXiv 7 Feb 2022.

Builds on

31 of 61 references.

References · 31

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  3. 2018Federated Learning with Non-IID Datacited by 1,108
  4. 2020Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimizationcited by 906
  5. 2016CVXPY: A Python-Embedded Modeling Language for Convex Optimizationcited by 641
  6. 2017Generative Adversarial Networks: An Overviewcited by 590
  7. 2018Adaptive Federated Learning in Resource Constrained Edge Computing Systemscited by 409
  8. 2019A Joint Learning and Communications Framework for Federated Learning over Wireless Networkscited by 263
  9. 2019Client-Edge-Cloud Hierarchical Federated Learningcited by 202
  10. 2018Broadband Analog Aggregation for Low-Latency Federated Edge Learning (Extended Version)cited by 191
  11. 2019Federated Learning over Wireless Fading Channelscited by 183
  12. 2018Towards an Intelligent Edge: Wireless Communication Meets Machine Learningcited by 95
19 more
  1. 2019Federated Learning over Wireless Networks: Convergence Analysis and Resource Allocationcited by 94
  2. 2018Wireless Network Intelligence at the Edgecited by 92
  3. 2019Federated Learning with Cooperating Devices: A Consensus Approach for Massive IoT Networkscited by 79
  4. 2020Towards Flexible Device Participation in Federated Learningcited by 65
  5. 2021Semi-Decentralized Federated Learning with Cooperative D2D Local Model Aggregationscited by 51
  6. 2018Wide Compression: Tensor Ring Netscited by 50
  7. 2020Wireless Communications for Collaborative Federated Learningcited by 47
  8. 2020Multi-Stage Hybrid Federated Learning over Large-Scale D2D-Enabled Fog Networkscited by 42
  9. 2021Device Sampling for Heterogeneous Federated Learning: Theory, Algorithms, and Implementationcited by 40
  10. 2018SparCML: High-Performance Sparse Communication for Machine Learningcited by 34
  11. 2020Decentralized Federated Learning via SGD over Wireless D2D Networkscited by 33
  12. 2020Network-Aware Optimization of Distributed Learning for Fog Computingcited by 27
  13. 2014Accelerating Minibatch Stochastic Gradient Descent using Stratified Samplingcited by 21
  14. 2021UAV-assisted Online Machine Learning over Multi-Tiered Networks: A Hierarchical Nested Personalized Federated Learning Approachcited by 20
  15. 2020Dynamic Sampling and Selective Masking for Communication-Efficient Federated Learningcited by 16
  16. 2020Coded Federated Learningcited by 14
  17. 2018Principled Deep Neural Network Training through Linear Programmingcited by 13
  18. 2020Dynamic Federated Learningcited by 12
  19. 2020Can we Generalize and Distribute Private Representation Learning?cited by 5

Built on by

Cited by year

Citing papers per year: 2022: 4; 2023: 10; 2024: 3; 2025: 1.

18 papers.

Sort:

  1. 2023Federated Learning for Connected and Automated Vehicles: A Survey of Existing Approaches and Challengescited by 40
  2. 2023Performative Federated Learning: A Solution to Model-Dependent and Heterogeneous Distribution Shiftscited by 14
  3. 2022Latency Optimization for Blockchain-Empowered Federated Learning in Multi-Server Edge Computingcited by 13
  4. 2022Multi-Edge Server-Assisted Dynamic Federated Learning with an Optimized Floating Aggregation Pointcited by 10
  5. 2022Online Federated Learning via Non-Stationary Detection and Adaptation amidst Concept Driftcited by 10
  6. 2023Delay-Aware Hierarchical Federated Learningcited by 9
  7. 2023Towards Cooperative Federated Learning over Heterogeneous Edge/Fog Networkscited by 8
  8. 2023Multi-Source to Multi-Target Decentralized Federated Domain Adaptationcited by 6
  9. 2022Embedding Alignment for Unsupervised Federated Learning via Smart Data Exchangecited by 5
  10. 2024Resource-Aware Hierarchical Federated Learning in Wireless Video Caching Networkscited by 4
  11. 2023Asynchronous Multi-Model Dynamic Federated Learning over Wireless Networks: Theory, Modeling, and Optimizationcited by 3
  12. 2023Device Sampling and Resource Optimization for Federated Learning in Cooperative Edge Networkscited by 2
6 more
  1. 2023Digital Over-the-Air Federated Learning in Multi-Antenna Systemscited by 1
  2. 2023Elastic Federated Learning over Open Radio Access Network (O-RAN) for Concurrent Execution of Multiple Distributed Learning Taskscited by 1
  3. 2023Resource-Aware Hierarchical Federated Learning for Video Caching in Wireless Networkscited by 1
  4. 2024Online-Score-Aided Federated Learning for Resource-Constrained Wireless Clients with Continual Data Arrivalcited by 1
  5. 2025The Transition from Centralized Machine Learning to Federated Learning for Mental Health in Education: A Survey of Current Methods and Future Directionscited by 1
  6. 2024Unsupervised Federated Optimization at the Edge: D2D-Enabled Learning without Labelscited by 0