mlsanityData to 7 Oct 2026

Fed-GraB: Federated Long-tailed Learning with Self-Adjusting Gradient Balancer

arXiv 2310.07587 ↗First posted 11 Oct 2023cs.LG cs.AIcited by 17

Not on the map. Why?

Abstract (arXiv)

Data privacy and long-tailed distribution are the norms rather than the exception in many real-world tasks. This paper investigates a federated long-tailed learning (Fed-LT) task in which each client holds a locally heterogeneous dataset; if the datasets can be globally aggregated, they jointly exhibit a long-tailed distribution. Under such a setting, existing federated optimization and/or centralized long-tailed learning methods hardly apply due to challenges in (a) characterizing the global long-tailed distribution under privacy constraints and (b) adjusting the local learning strategy to cope with the head-tail imbalance. In response, we propose a method termed textttFed-GraB, comprised of a Self-adjusting Gradient Balancer (SGB) module that re-weights clients' gradients in a closed-loop manner, based on the feedback of global long-tailed distribution evaluated by a Direct Prior Analyzer (DPA) module. Using textttFed-GraB, clients can effectively alleviate the distribution drift caused by data heterogeneity during the model training process and obtain a global model with better performance on the minority classes while maintaining the performance of the majority classes. Extensive experiments demonstrate that textttFed-GraB achieves state-of-the-art performance on representative datasets such as CIFAR-10-LT, CIFAR-100-LT, ImageNet-LT, and iNaturalist.

Key facts. Cited by 17 papers in the map (data through 7 Oct 2026). Builds on 45 of its 62 references that are in the map. First posted on arXiv 11 Oct 2023.

Builds on

45 of 62 references.

References · 45

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  3. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  4. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  5. 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,531
  6. 2020Adaptive Federated Optimizationcited by 1,036
  7. 2019Learning Imbalanced Datasets with Label-Distribution-Aware Margin Losscited by 987
  8. 2020Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimizationcited by 897
  9. 2019Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classificationcited by 836
  10. 2019Decoupling Representation and Classifier for Long-Tailed Recognitioncited by 806
  11. 2021Model-Contrastive Federated Learningcited by 762
  12. 2019Large-Scale Long-Tailed Recognition in an Open Worldcited by 704
33 more
  1. 2020Personalized Federated Learning with Moreau Envelopescited by 612
  2. 2021Federated Learning on Non-IID Data Silos: An Experimental Studycited by 566
  3. 2021Federated Learning Based on Dynamic Regularizationcited by 555
  4. 2021Towards Personalized Federated Learningcited by 462
  5. 2021Deep Long-Tailed Learning: A Surveycited by 416
  6. 2020Equalization Loss for Long-Tailed Object Recognitioncited by 285
  7. 2021No Fear of Heterogeneity: Classifier Calibration for Federated Learning with Non-IID Datacited by 241
  8. 2021Evaluating Gradient Inversion Attacks and Defenses in Federated Learningcited by 196
  9. 2020Overcoming Classifier Imbalance for Long-tail Object Detection with Balanced Group Softmaxcited by 152
  10. 2020Seesaw Loss for Long-Tailed Instance Segmentationcited by 146
  11. 2020Federated Visual Classification with Real-World Data Distributioncited by 126
  12. 2020Distribution-Balanced Loss for Multi-Label Classification in Long-Tailed Datasetscited by 122
  13. 2020Deep Representation Learning on Long-tailed Data: A Learnable Embedding Augmentation Perspectivecited by 120
  14. 2020The Devil is in Classification: A Simple Framework for Long-tail Object Detection and Instance Segmentationcited by 108
  15. 2021FedMix: Approximation of Mixup under Mean Augmented Federated Learningcited by 108
  16. 2022Federated Learning with Label Distribution Skew via Logits Calibrationcited by 108
  17. 2022Inducing Neural Collapse in Imbalanced Learning: Do We Really Need a Learnable Classifier at the End of Deep Neural Network?cited by 100
  18. 2020Equalization Loss v2: A New Gradient Balance Approach for Long-tailed Object Detectioncited by 99
  19. 2021MetaSAug: Meta Semantic Augmentation for Long-Tailed Visual Recognitioncited by 99
  20. 2021ACE: Ally Complementary Experts for Solving Long-Tailed Recognition in One-Shotcited by 86
  21. 2021FASA: Feature Augmentation and Sampling Adaptation for Long-Tailed Instance Segmentationcited by 76
  22. 2022A Survey on Long-Tailed Visual Recognitioncited by 75
  23. 2023Long-tailed Visual Recognition via Gaussian Clouded Logit Adjustmentcited by 75
  24. 2022Federated Learning on Heterogeneous and Long-Tailed Data via Classifier Re-Training with Federated Featurescited by 63
  25. 2021RSG: A Simple but Effective Module for Learning Imbalanced Datasetscited by 61
  26. 2022FedCorr: Multi-Stage Federated Learning for Label Noise Correctioncited by 61
  27. 2020LotteryFL: Personalized and Communication-Efficient Federated Learning with Lottery Ticket Hypothesis on Non-IID Datasetscited by 60
  28. 2021On Exposing the Challenging Long Tail in Future Prediction of Traffic Actorscited by 48
  29. 2022Equalized Focal Loss for Dense Long-Tailed Object Detectioncited by 46
  30. 2021GistNet: a Geometric Structure Transfer Network for Long-Tailed Recognitioncited by 31
  31. 2021Dynamic Attention-based Communication-Efficient Federated Learningcited by 9
  32. 2022Towards Federated Long-Tailed Learningcited by 7
  33. 2023Personalized Federated Learning on Long-Tailed Data via Adversarial Feature Augmentationcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 5; 2025: 7; 2026: 5.

17 papers.

Sort:

By area:

  1. 2025Self-Evolving Multi-Agent Systems via Textual Backpropagationcited by 21
  2. 2025Balancing the Scales: A Theoretical and Algorithmic Framework for Learning from Imbalanced Datacited by 15
  3. 2024Exploring Vacant Classes in Label-Skewed Federated Learningcited by 10
  4. 2024FedLoGe: Joint Local and Generic Federated Learning under Long-tailed Datacited by 10
  5. 2025Improved Balanced Classification with Theoretically Grounded Loss Functionscited by 9
  6. 2025You Are Your Own Best Teacher: Achieving Centralized-level Performance in Federated Learning under Heterogeneous and Long-tailed Datacited by 5
  7. 2024Adaptive Federated Learning Over the Aircited by 4
  8. 2024Addressing Skewed Heterogeneity via Federated Prototype Rectification with Personalizationcited by 2
  9. 2024Overcoming label shift with target-aware federated learningcited by 2
  10. 2025CAPT: Class-Aware Prompt Tuning for Federated Long-Tailed Learning with Vision-Language Modelcited by 1
  11. 2025AdRo-FL: Informed and Secure Client Selection for Federated Learning in the Presence of Adversarial Aggregatorcited by 1
  12. 2026Fine-Tuning Impairs the Balancedness of Foundation Models in Long-tailed Personalized Federated Learningcited by 1
5 more
  1. 2025FedWCM: Unleashing the Potential of Momentum-based Federated Learning in Long-Tailed Scenarioscited by 0
  2. 2026Towards Federated Long-Tailed Graph Learning: An Energy-Guided Dual Decoupling Approachcited by 0
  3. 2026FedReLa: Imbalanced Federated Learning via Re-Labelingcited by 0
  4. 2026Class-Grouped Normalized Momentum and Faster Hyperparameter Exploration to Tackle Class Imbalance in Federated Learningcited by 0
  5. 2026FedLTLib: A Comprehensive Benchmark for Federated Long-Tail Learningcited by 0