mlsanityData to 14 Oct 2026

Range Penalization: Theoretical Insights with Applications in Federated Learning

arXiv 2606.10916 ↗First posted 9 Jun 2026stat.ML cs.LG math.STcited by 0

Not on the map. Why?

Abstract (arXiv)

This paper introduces range regularization for federated learning with linear systematic components to enhance statistical accuracy and induce cross-client regularity conducive to quantization, coding, and resource efficiency. Our approach identifies features with shared weights across different clients and adaptively clusters the weights of personalized features at extreme values, a process we refer to as polar clustering. Theoretical analysis of the associated estimators poses significant challenges due to the seminorm nature and non-decomposability of the regularizer. We develop new proof techniques for the nonasymptotic analysis of statistical accuracy and faithful pattern recovery. Moreover, a fast optimization algorithm that leverages varying degrees of local strong convexity is proposed to reduce iteration complexity. Experiments support the efficacy and efficiency of the proposed approach.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 24 of its 70 references that are in the map. First posted on arXiv 9 Jun 2026.

Builds on

24 of 70 references.

References · 24

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  2. 2018Federated Optimization in Heterogeneous Networkscited by 2,987
  3. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,329
  4. 2019Federated Learning with Personalization Layerscited by 541
  5. 2020An Efficient Framework for Clustered Federated Learningcited by 511
  6. 2019Robust and Communication-Efficient Federated Learning from Non-IID Datacited by 419
  7. 2020Adaptive Personalized Federated Learningcited by 358
  8. 2019FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantizationcited by 356
  9. 2018Client Selection for Federated Learning with Heterogeneous Resources in Mobile Edgecited by 352
  10. 2019Improving Federated Learning Personalization via Model Agnostic Meta Learningcited by 269
  11. 2019Model Pruning Enables Efficient Federated Learning on Edge Devicescited by 222
  12. 2015The Benefit of Multitask Representation Learningcited by 195
12 more
  1. 2022ProxSkip: Yes! Local Gradient Steps Provably Lead to Communication Acceleration! Finally!cited by 134
  2. 2022Federated Learning with Partial Model Personalizationcited by 114
  3. 2013Regularized M-estimators with nonconvexity: Statistical and algorithmic theory for local optimacited by 66
  4. 2020Personalized Federated Learning for Intelligent IoT Applications: A Cloud-Edge based Frameworkcited by 59
  5. 2022Motley: Benchmarking Heterogeneity and Personalization in Federated Learningcited by 31
  6. 2013Splitting Methods for Convex Clusteringcited by 28
  7. 2021Towards Energy Efficient Federated Learning over 5G+ Mobile Devicescited by 16
  8. 2023FedHB: Hierarchical Bayesian Federated Learningcited by 8
  9. 2018On Cross-validation for Sparse Reduced Rank Regressioncited by 6
  10. 2021Analysis of Generalized Bregman Surrogate Algorithms for Nonsmooth Nonconvex Statistical Learningcited by 5
  11. 2021Supervised Multivariate Learning with Simultaneous Feature Auto-grouping and Dimension Reductioncited by 3
  12. 2022Dual Seminorms, Ergodic Coefficients and Semicontraction Theorycited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2023Communication-Efficient Federated Learning via Regularized Sparse Random Networks
  2. 2023Adaptive Self-Distillation for Minimizing Client Drift in Heterogeneous Federated Learning
  3. 2025OvA-LP: A Simple and Efficient Framework for Federated Learning on Non-IID Data
  4. 2022Multi-Level Branched Regularization for Federated Learning
  5. 2024Tailored Federated Learning: Leveraging Direction Regulation & Knowledge Distillation
  6. 2022Communication-Efficient Federated Learning With Data and Client Heterogeneity
  7. 2024ADEPT: Hierarchical Bayes Approach to Personalized Federated Unsupervised Learning
  8. 2022Towards Federated Learning against Noisy Labels via Local Self-Regularization