mlsanityData to 7 Oct 2026

Smoothing ADMM for Non-convex and Non-smooth Hierarchical Federated Learning

arXiv 2503.08869 ↗First posted 11 Mar 2025cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

This paper presents a hierarchical federated learning (FL) framework that extends the alternating direction method of multipliers (ADMM) with smoothing techniques, tailored for non-convex and non-smooth objectives. Unlike traditional hierarchical FL methods, our approach supports asynchronous updates and multiple updates per iteration, enhancing adaptability to heterogeneous data and system settings. Additionally, we introduce a flexible mechanism to leverage diverse regularization functions at each layer, allowing customization to the specific prior information within each cluster and accommodating (possibly) non-smooth penalty objectives. Depending on the learning goal, the framework supports both consensus and personalization: the total variation norm can be used to enforce consensus across layers, while non-convex penalties such as minimax concave penalty (MCP) or smoothly clipped absolute deviation (SCAD) enable personalized learning. Experimental results demonstrate the superior convergence rates and accuracy of our method compared to conventional approaches, underscoring its robustness and versatility for a wide range of FL scenarios.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 20 of its 32 references that are in the map. First posted on arXiv 11 Mar 2025.

Builds on

20 of 32 references.

References · 20

  1. 2019Advances and Open Problems in Federated Learningcited by 3,021
  2. 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,531
  3. 2017Federated Multi-Task Learningcited by 669
  4. 2019FedMD: Heterogenous Federated Learning via Model Distillationcited by 454
  5. 2018Client Selection for Federated Learning with Heterogeneous Resources in Mobile Edgecited by 347
  6. 2020HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clientscited by 334
  7. 2021Federated Multi-Task Learning under a Mixture of Distributionscited by 189
  8. 2022Federated Learning with Label Distribution Skew via Logits Calibrationcited by 108
  9. 2015Global Convergence of ADMM in Nonconvex Nonsmooth Optimizationcited by 98
  10. 2014Convergence Analysis of Alternating Direction Method of Multipliers for a Family of Nonconvex Problemscited by 64
  11. 2017Network Inference via the Time-Varying Graphical Lassocited by 55
  12. 2021Hierarchical Federated Learning with Quantization: Convergence Analysis and System Designcited by 47
8 more
  1. 2022FedBalancer: Data and Pace Control for Efficient Federated Learning on Heterogeneous Clientscited by 46
  2. 2018Subgradient methods for sharp weakly convex functionscited by 33
  3. 2017Douglas-Rachford splitting and ADMM for nonconvex optimization: tight convergence resultscited by 21
  4. 2021Clustered Federated Learning via Generalized Total Variation Minimizationcited by 7
  5. 2022Clustered Federated Learning based on Nonconvex Pairwise Fusioncited by 4
  6. 2023Smoothing ADMM for Sparse-Penalized Quantile Regression with Non-Convex Penaltiescited by 4
  7. 2024Hierarchical Federated ADMMcited by 4
  8. 2024Decentralized Smoothing ADMM for Quantile Regression with Non-Convex Sparse Penaltiescited by 2

Built on by

Not cited yet.