mlsanityData to 7 Oct 2026

Extra Clients at No Extra Cost: Overcome Data Heterogeneity in Federated Learning with Filter Decomposition

arXiv 2503.08652 ↗First posted 11 Mar 2025cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Data heterogeneity is one of the major challenges in federated learning (FL), which results in substantial client variance and slow convergence. In this study, we propose a novel solution: decomposing a convolutional filter in FL into a linear combination of filter subspace elements, i.e., filter atoms. This simple technique transforms global filter aggregation in FL into aggregating filter atoms and their atom coefficients. The key advantage here involves mathematically generating numerous cross-terms by expanding the product of two weighted sums from filter atom and atom coefficient. These cross-terms effectively emulate many additional latent clients, significantly reducing model variance, which is validated by our theoretical analysis and empirical observation. Furthermore, our method permits different training schemes for filter atoms and atom coefficients for highly adaptive model personalization and communication efficiency. Empirical results on benchmark datasets demonstrate that our filter decomposition technique substantially improves the accuracy of FL methods, confirming its efficacy in addressing data heterogeneity.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 34 of its 54 references that are in the map. First posted on arXiv 11 Mar 2025.

Builds on

34 of 54 references.

References · 34

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  3. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  4. 2019On the Convergence of FedAvg on Non-IID Datacited by 1,220
  5. 2017Visualizing the Loss Landscape of Neural Netscited by 1,126
  6. 2017A Survey on Multi-Task Learningcited by 838
  7. 2021Model-Contrastive Federated Learningcited by 762
  8. 2017Federated Multi-Task Learningcited by 669
  9. 2020Ditto: Fair and Robust Federated Learning Through Personalizationcited by 665
  10. 2020Personalized Federated Learning with Moreau Envelopescited by 612
  11. 2021Federated Learning Based on Dynamic Regularizationcited by 555
  12. 2021Exploiting Shared Representations for Personalized Federated Learningcited by 537
22 more
  1. 2019Federated Learning with Personalization Layerscited by 533
  2. 2020An Efficient Framework for Clustered Federated Learningcited by 505
  3. 2019Clustered Federated Learning: Model-Agnostic Distributed Multi-Task Optimization under Privacy Constraintscited by 499
  4. 2019Bayesian Nonparametric Federated Learning of Neural Networkscited by 385
  5. 2020Think Locally, Act Globally: Federated Learning with Local and Global Representationscited by 322
  6. 2020Personalized Cross-Silo Federated Learning on Non-IID Datacited by 282
  7. 2020Personalized Federated Learning with First Order Model Optimizationcited by 201
  8. 2019Adaptive Gradient-Based Meta-Learning Methodscited by 183
  9. 2022FedDC: Federated Learning with Non-IID Data via Local Drift Decoupling and Correctioncited by 181
  10. 2021FedMix: Approximation of Mixup under Mean Augmented Federated Learningcited by 108
  11. 2023Personalized Federated Learning with Feature Alignment and Classifier Collaborationcited by 97
  12. 2018D"IoT: A Federated Self-learning Anomaly Detection System for IoTcited by 89
  13. 2021FedProc: Prototypical Contrastive Federated Learning on Non-IID datacited by 80
  14. 2021Personalized Federated Learning with Gaussian Processescited by 76
  15. 2022FedVARP: Tackling the Variance Due to Partial Client Participation in Federated Learningcited by 62
  16. 2020What Can Be Transferred: Unsupervised Domain Adaptation for Endoscopic Lesions Segmentationcited by 50
  17. 2021Flexible Clustered Federated Learning for Client-Level Data Distribution Shiftcited by 47
  18. 2021FLOP: Federated Learning on Medical Datasets using Partial Networkscited by 42
  19. 2022Variance Reduced ProxSkip: Algorithm, Theory and Application to Federated Learningcited by 33
  20. 2018DCFNet: Deep Neural Network with Decomposed Convolutional Filterscited by 23
  21. 2020New Directions in Distributed Deep Learning: Bringing the Network at Forefront of IoT Designcited by 3
  22. 2021TESSERACT: Gradient Flip Score to Secure Federated Learning Against Model Poisoning Attackscited by 3

Built on by

Not cited yet.