mlsanityData to 7 Oct 2026

FedLoGe: Joint Local and Generic Federated Learning under Long-tailed Data

arXiv 2401.08977 ↗First posted 17 Jan 2024cs.LG cs.AIcited by 10

Not on the map. Why?

Abstract (arXiv)

Federated Long-Tailed Learning (Fed-LT), a paradigm wherein data collected from decentralized local clients manifests a globally prevalent long-tailed distribution, has garnered considerable attention in recent times. In the context of Fed-LT, existing works have predominantly centered on addressing the data imbalance issue to enhance the efficacy of the generic global model while neglecting the performance at the local level. In contrast, conventional Personalized Federated Learning (pFL) techniques are primarily devised to optimize personalized local models under the presumption of a balanced global data distribution. This paper introduces an approach termed Federated Local and Generic Model Training in Fed-LT (FedLoGe), which enhances both local and generic model performance through the integration of representation learning and classifier alignment within a neural collapse framework. Our investigation reveals the feasibility of employing a shared backbone as a foundational framework for capturing overarching global trends, while concurrently employing individualized classifiers to encapsulate distinct refinements stemming from each client's local features. Building upon this discovery, we establish the Static Sparse Equiangular Tight Frame Classifier (SSE-C), inspired by neural collapse principles that naturally prune extraneous noisy features and foster the acquisition of potent data representations. Furthermore, leveraging insights from imbalance neural collapse's classifier norm patterns, we develop Global and Local Adaptive Feature Realignment (GLA-FR) via an auxiliary global classifier and personalized Euclidean norm transfer to align global features with client preferences. Extensive experimental results on CIFAR-10/100-LT, ImageNet, and iNaturalist demonstrate the advantage of our method over state-of-the-art pFL and Fed-LT approaches.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 60 of its 74 references that are in the map. First posted on arXiv 17 Jan 2024.

Builds on

60 of 74 references.

References · 60

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  2. 2019Advances and Open Problems in Federated Learningcited by 3,021
  3. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,250
  4. 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,531
  5. 2019Learning Imbalanced Datasets with Label-Distribution-Aware Margin Losscited by 987
  6. 2019Decoupling Representation and Classifier for Long-Tailed Recognitioncited by 806
  7. 2020Prevalence of Neural Collapse during the terminal phase of deep learning trainingcited by 713
  8. 2019Large-Scale Long-Tailed Recognition in an Open Worldcited by 704
  9. 2020Ditto: Fair and Robust Federated Learning Through Personalizationcited by 665
  10. 2020Personalized Federated Learning with Moreau Envelopescited by 612
  11. 2021FedBN: Federated Learning on Non-IID Features via Local Batch Normalizationcited by 565
  12. 2021Exploiting Shared Representations for Personalized Federated Learningcited by 537
38 more
  1. 2019Federated Learning with Personalization Layerscited by 533
  2. 2020An Efficient Framework for Clustered Federated Learningcited by 505
  3. 2019Clustered Federated Learning: Model-Agnostic Distributed Multi-Task Optimization under Privacy Constraintscited by 499
  4. 2021Towards Personalized Federated Learningcited by 462
  5. 2019FedMD: Heterogenous Federated Learning via Model Distillationcited by 454
  6. 2020Adaptive Personalized Federated Learningcited by 353
  7. 2020HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clientscited by 334
  8. 2021A Geometric Analysis of Neural Collapse with Unconstrained Featurescited by 226
  9. 2020Federated learning with hierarchical clustering of local updates to improve training on non-IID datacited by 222
  10. 2021Evaluating Gradient Inversion Attacks and Defenses in Federated Learningcited by 196
  11. 2019FedHealth: A Federated Transfer Learning Framework for Wearable Healthcarecited by 182
  12. 2021FedBABU: Towards Enhanced Representation for Federated Image Classificationcited by 180
  13. 2021Exploring Deep Neural Networks via Layer-Peeled Model: Minority Collapse in Imbalanced Trainingcited by 166
  14. 2020Overcoming Classifier Imbalance for Long-tail Object Detection with Balanced Group Softmaxcited by 152
  15. 2020Neural collapse with unconstrained featurescited by 140
  16. 2022Federated Learning with Partial Model Personalizationcited by 114
  17. 2023Neural Collapse Inspired Feature-Classifier Alignment for Few-Shot Class Incremental Learningcited by 112
  18. 2022Federated Learning with Label Distribution Skew via Logits Calibrationcited by 108
  19. 2022Extended Unconstrained Features Model for Exploring Deep Neural Collapsecited by 101
  20. 2022Inducing Neural Collapse in Imbalanced Learning: Do We Really Need a Learnable Classifier at the End of Deep Neural Network?cited by 100
  21. 2020Equalization Loss v2: A New Gradient Balance Approach for Long-tailed Object Detectioncited by 99
  22. 2021An Unconstrained Layer-Peeled Perspective on Neural Collapsecited by 87
  23. 2022Imbalance Trouble: Revisiting Neural-Collapse Geometrycited by 80
  24. 2023Long-tailed Visual Recognition via Gaussian Clouded Logit Adjustmentcited by 75
  25. 2022DisPFL: Towards Communication-Efficient Personalized Federated Learning via Decentralized Sparse Trainingcited by 74
  26. 2022Personalized Federated Learning via Variational Bayesian Inferencecited by 68
  27. 2022Tackling Data Heterogeneity in Federated Learning with Class Prototypescited by 66
  28. 2022A Broad Study of Pre-training for Domain Generalization and Adaptationcited by 64
  29. 2022Federated Learning on Heterogeneous and Long-Tailed Data via Classifier Re-Training with Federated Featurescited by 63
  30. 2023No Fear of Classifier Biases: Neural Collapse Inspired Federated Learning with Synthetic and Fixed Classifiercited by 55
  31. 2019Adjusting Decision Boundary for Class Imbalanced Learningcited by 52
  32. 2020Federated learning with class imbalance reductioncited by 52
  33. 2022Neural Collapse with Normalized Features: A Geometric Analysis over the Riemannian Manifoldcited by 50
  34. 2023Understanding Imbalanced Semantic Segmentation Through Neural Collapsecited by 46
  35. 2023Neural Collapse in Deep Linear Networks: From Balanced to Imbalanced Datacited by 40
  36. 2022CD2-pFed: Cyclic Distillation-guided Channel Decoupling for Model Personalization in Federated Learningcited by 34
  37. 2022Fed-CBS: A Heterogeneity-Aware Client Sampling Mechanism for Federated Learning via Class-Imbalance Reductioncited by 32
  38. 2022Understanding and Improving Transfer Learning of Deep Models via Neural Collapsecited by 24

50 most cited of 60 All 60 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 3; 2026: 5.

10 papers.

  1. 2025Self-Evolving Multi-Agent Systems via Textual Backpropagationcited by 21
  2. 2025You Are Your Own Best Teacher: Achieving Centralized-level Performance in Federated Learning under Heterogeneous and Long-tailed Datacited by 5
  3. 2024FedLF: Adaptive Logit Adjustment and Feature Optimization in Federated Long-Tailed Learningcited by 4
  4. 2024FedDr+: Stabilizing Dot-regression with Global Feature Distillation for Federated Learningcited by 2
  5. 2025CAPT: Class-Aware Prompt Tuning for Federated Long-Tailed Learning with Vision-Language Modelcited by 1
  6. 2026Fine-Tuning Impairs the Balancedness of Foundation Models in Long-tailed Personalized Federated Learningcited by 1
  7. 2026FedReLa: Imbalanced Federated Learning via Re-Labelingcited by 0
  8. 2026FedLTLib: A Comprehensive Benchmark for Federated Long-Tail Learningcited by 0
  9. 2026Personalized Federated Learning through Global Knowledge Distillation and Local Head Adaptationcited by 0
  10. 2026Understanding Head Geometry and Dynamics in Federated Regression through a Natural Solution Selection Rule: An Unconstrained Feature Model Analysiscited by 0