mlsanityData to 14 Oct 2026

Towards Serverless Semi-Decentralized Federated Learning with Heterogeneous Optimizers

arXiv 2606.06687 ↗First posted 4 Jun 2026cs.LG cs.DC cs.NIcited by 0

Not on the map. Why?

Abstract (arXiv)

We investigate cluster formation, involving the number and composition of clusters, in decentralized federated learning (FL) with heterogeneous machine learning (ML) optimizers. While clustering in centralized FL has enabled scalability and resource savings, its value and development in fully decentralized environments have yet to be explored. Optimizing cluster formation in such environments is challenging, especially due to the complex coupling between network graph structures, local data heterogeneity, and different local ML model optimizers. To address these challenges, we propose serverless semi-decentralized FL (SSD-FL), a methodology requiring no persistent server infrastructure. In SSD-FL, cluster formation occurs via a lightweight, one-time device-to-device (D2D) initialization phase, after which actual ML model training (alongside consensus and convergence processes) is fully serverless. Functionally, SSD-FL segments global rounds into intra-cluster and inter-cluster regimes, ensuring global convergence and consensus through novel "effective loss functions" that integrate device-specific ML optimizers with network graph-based regularization. Next, SSD-FL leverages the consensus gap via the Cheeger inequality to develop an iterative clustering algorithm evaluated against our derived convergence and consensus bounds, which incorporate a unique scoring metric to quantify data and optimizer heterogeneity across devices. Finally, experimental evaluation against three categories of decentralized FL methodologies validate that SSD-FL improves both convergence speeds and communication efficiency across various network graphs, datasets, and local optimizer regimes.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 27 of its 55 references that are in the map. First posted on arXiv 4 Jun 2026.

Builds on

27 of 55 references.

References · 27

  1. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,349
  2. 2018Adaptive Federated Learning in Resource Constrained Edge Computing Systemscited by 409
  3. 2020A Unified Theory of Decentralized SGD with Changing Topology and Local Updatescited by 367
  4. 2014EXTRA: An Exact First-Order Algorithm for Decentralized Consensus Optimizationcited by 160
  5. 2021Decentralized Federated Averagingcited by 114
  6. 2023Decentralized Federated Learning: A Survey and Perspectivecited by 86
  7. 2013Distributed optimization over time-varying directed graphscited by 78
  8. 2014A Proximal Stochastic Gradient Method with Progressive Variance Reductioncited by 78
  9. 2017Collaborative Deep Learning in Fixed Topology Networkscited by 60
  10. 2021Decentralized Federated Learning: Balancing Communication and Computing Costscited by 55
  11. 2021Semi-Decentralized Federated Learning with Cooperative D2D Local Model Aggregationscited by 51
  12. 2021Hierarchical Federated Learning with Quantization: Convergence Analysis and System Designcited by 47
15 more
  1. 2024Decentralized Federated Learning: A Survey on Security and Privacycited by 25
  2. 2020Quantized Decentralized Stochastic Learning over Directed Graphscited by 24
  3. 2022SWIFT: Rapid Decentralized Federated Learning via Wait-Free Model Communicationcited by 21
  4. 2021UAV-assisted Online Machine Learning over Multi-Tiered Networks: A Hierarchical Nested Personalized Federated Learning Approachcited by 20
  5. 2022Semi-Decentralized Federated Learning with Collaborative Relayingcited by 16
  6. 2021Semi-Decentralized Federated Edge Learning for Fast Convergence on Non-IID Datacited by 12
  7. 2024Decentralized Sporadic Federated Learning: A Unified Algorithmic Framework with Convergence Guaranteescited by 10
  8. 2022On the complexity of finding stationary points of smooth functions in one dimensioncited by 8
  9. 2023Towards Cooperative Federated Learning over Heterogeneous Edge/Fog Networkscited by 8
  10. 2024Hierarchical Federated Learning in Multi-hop Cluster-Based VANETscited by 7
  11. 2020Peer-to-Peer Trading in Electricity Networks: An Overviewcited by 6
  12. 2024Decentralized Federated Learning with Gradient Tracking over Time-Varying Directed Networkscited by 3
  13. 2023Device Sampling and Resource Optimization for Federated Learning in Cooperative Edge Networkscited by 2
  14. 2025SDFLMQ: A Semi-Decentralized Federated Learning Framework over MQTTcited by 2
  15. 2025Hierarchical Federated Learning for Social Network with Mobilitycited by 2

Built on by

Not cited yet.