FedWCM: Unleashing the Potential of Momentum-based Federated Learning in Long-Tailed Scenarios
Not on the map. Why?
Abstract (arXiv)
Federated Learning (FL) enables decentralized model training while preserving data privacy. Despite its benefits, FL faces challenges with non-identically distributed (non-IID) data, especially in long-tailed scenarios with imbalanced class samples. Momentum-based FL methods, often used to accelerate FL convergence, struggle with these distributions, resulting in biased models and making FL hard to converge. To understand this challenge, we conduct extensive investigations into this phenomenon, accompanied by a layer-wise analysis of neural network behavior. Based on these insights, we propose FedWCM, a method that dynamically adjusts momentum using global and per-round data to correct directional biases introduced by long-tailed distributions. Extensive experiments show that FedWCM resolves non-convergence issues and outperforms existing methods, enhancing FL's efficiency and effectiveness in handling client heterogeneity and data imbalance.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 28 of its 60 references that are in the map. First posted on arXiv 20 Jul 2025.
Builds on
28 of 60 references.
References · 28
- 2015Deep Residual Learning for Image Recognitioncited by 52,096
- 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
- 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,349
- 2018Federated Optimization in Heterogeneous Networkscited by 2,987
- 2020Adaptive Federated Optimizationcited by 1,046
- 2019Learning Imbalanced Datasets with Label-Distribution-Aware Margin Losscited by 989
- 2021Federated Learning Based on Dynamic Regularizationcited by 562
- 2019Clustered Federated Learning: Model-Agnostic Distributed Multi-Task Optimization under Privacy Constraintscited by 503
- 2021Federated Learning on Non-IID Data: A Surveycited by 346
- 2020Long-Tailed Classification by Keeping the Good and Removing the Bad Momentum Causal Effectcited by 256
- 2018Communication-Efficient On-Device Machine Learning: Federated Distillation and Augmentation under Non-IID Private Datacited by 243
- 2020Disentangling Label Distribution for Long-tailed Visual Recognitioncited by 172
16 more
- 2021Exploring Deep Neural Networks via Layer-Peeled Model: Minority Collapse in Imbalanced Trainingcited by 166
- 2020Mime: Mimicking Centralized Stochastic Algorithms in Federated Learningcited by 154
- 2019SlowMo: Improving Communication-Efficient Distributed SGD with Slow Momentumcited by 121
- 2022Neural Collapse: A Review on Modelling Principles and Generalizationcited by 95
- 2021FedCM: Federated Learning with Client-level Momentumcited by 77
- 2019Astraea: Self-balancing Federated Learning for Improving Classification Accuracy of Mobile Deep Learning Applicationscited by 64
- 2022Federated Learning on Heterogeneous and Long-Tailed Data via Classifier Re-Training with Federated Featurescited by 63
- 2020Federated Learning via Synthetic Datacited by 54
- 2023Dynamic Regularized Sharpness Aware Minimization in Federated Learning: Approaching Global Consistency and Smooth Landscapecited by 46
- 2023Momentum Benefits Non-IID Federated Learning Simply and Provablycited by 38
- 2024Locally Estimated Global Perturbations are Better than Local Perturbations for Federated Sharpness-aware Minimizationcited by 28
- 2023Fed-GraB: Federated Long-tailed Learning with Self-Adjusting Gradient Balancercited by 17
- 2023On the Role of Server Momentum in Federated Learningcited by 16
- 2023A Survey on Class Imbalance in Federated Learningcited by 13
- 2023CLIP-guided Federated Learning on Heterogeneous and Long-Tailed Datacited by 9
- 2021Dubhe: Towards Data Unbiasedness with Homomorphic Encryption in Federated Learning Client Selectioncited by 7
Built on by
Citing papers per year: 2026: 1.
1 paper.