mlsanityData to 13 Oct 2026

No Fear of Classifier Biases: Neural Collapse Inspired Federated Learning with Synthetic and Fixed Classifier

arXiv 2303.10058 ↗First posted 17 Mar 2023cs.LG cs.CVcited by 55

Not on the map. Why?

Abstract (arXiv)

Data heterogeneity is an inherent challenge that hinders the performance of federated learning (FL). Recent studies have identified the biased classifiers of local models as the key bottleneck. Previous attempts have used classifier calibration after FL training, but this approach falls short in improving the poor feature representations caused by training-time classifier biases. Resolving the classifier bias dilemma in FL requires a full understanding of the mechanisms behind the classifier. Recent advances in neural collapse have shown that the classifiers and feature prototypes under perfect training scenarios collapse into an optimal structure called simplex equiangular tight frame (ETF). Building on this neural collapse insight, we propose a solution to the FL's classifier bias problem by utilizing a synthetic and fixed ETF classifier during training. The optimal classifier structure enables all clients to learn unified and optimal feature representations even under extremely heterogeneous data. We devise several effective modules to better adapt the ETF structure in FL, achieving both high generalization and personalization. Extensive experiments demonstrate that our method achieves state-of-the-art performances on CIFAR-10, CIFAR-100, and Tiny-ImageNet.

Key facts. Cited by 55 papers in the map (data through 13 Oct 2026). Builds on 41 of its 58 references that are in the map. First posted on arXiv 17 Mar 2023.

Builds on

41 of 58 references.

References · 41

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,950
  2. 2015Distilling the Knowledge in a Neural Networkcited by 10,180
  3. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,653
  4. 2016Densely Connected Convolutional Networkscited by 6,707
  5. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,426
  6. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,786
  7. 2018Federated Optimization in Heterogeneous Networkscited by 2,981
  8. 2017Visualizing the Loss Landscape of Neural Netscited by 1,129
  9. 2020Prevalence of Neural Collapse during the terminal phase of deep learning trainingcited by 716
  10. 2020Ensemble Distillation for Robust Model Fusion in Federated Learningcited by 682
  11. 2020Ditto: Fair and Robust Federated Learning Through Personalizationcited by 668
  12. 2021Federated Learning Based on Dynamic Regularizationcited by 559
29 more
  1. 2021Exploiting Shared Representations for Personalized Federated Learningcited by 539
  2. 2020An Efficient Framework for Clustered Federated Learningcited by 508
  3. 2020Balanced Meta-Softmax for Long-Tailed Visual Recognitioncited by 399
  4. 2021FedProto: Federated Prototype Learning across Heterogeneous Clientscited by 352
  5. 2021Federated Learning for Internet of Things: A Comprehensive Surveycited by 251
  6. 2021No Fear of Heterogeneity: Classifier Calibration for Federated Learning with Non-IID Datacited by 241
  7. 2021A Geometric Analysis of Neural Collapse with Unconstrained Featurescited by 226
  8. 2020Federated Learning for Internet of Things: Recent Advances, Taxonomy, and Open Challengescited by 139
  9. 2022FedALA: Adaptive Local Aggregation for Personalized Federated Learningcited by 122
  10. 2023Neural Collapse Inspired Feature-Classifier Alignment for Few-Shot Class Incremental Learningcited by 115
  11. 2022Federated Learning with Label Distribution Skew via Logits Calibrationcited by 109
  12. 2022Extended Unconstrained Features Model for Exploring Deep Neural Collapsecited by 102
  13. 2021DENSE: Data-Free One-Shot Federated Learningcited by 100
  14. 2022Neural Collapse: A Review on Modelling Principles and Generalizationcited by 95
  15. 2019Reliable Federated Learning for Mobile Networkscited by 92
  16. 2021An Unconstrained Layer-Peeled Perspective on Neural Collapsecited by 87
  17. 2022Imbalance Trouble: Revisiting Neural-Collapse Geometrycited by 80
  18. 2022Tackling Data Heterogeneity in Federated Learning with Class Prototypescited by 67
  19. 2022Federated Learning on Heterogeneous and Long-Tailed Data via Classifier Re-Training with Federated Featurescited by 63
  20. 2021What Do We Mean by Generalization in Federated Learning?cited by 62
  21. 2023Revisiting Weighted Aggregation in Federated Learning with Neural Networkscited by 58
  22. 2022FedFA: Federated Learning with Feature Anchors to Align Features and Classifiers for Heterogeneous Datacited by 50
  23. 2022Neural Collapse Inspired Attraction-Repulsion-Balanced Loss for Imbalanced Learningcited by 43
  24. 2020Training Keyword Spotting Models on Non-IID Data with Federated Learningcited by 36
  25. 2022FedFM: Anchor-based Feature Matching for Data Heterogeneity in Federated Learningcited by 27
  26. 2022Towards Effective Clustered Federated Learning: A Peer-to-peer Framework with Adaptive Neighbor Matchingcited by 24
  27. 2022Understanding and Improving Transfer Learning of Deep Models via Neural Collapsecited by 24
  28. 2022Auto-FedRL: Federated Hyperparameter Optimization for Multi-institutional Medical Image Segmentationcited by 16
  29. 2022On the effectiveness of partial variance reduction in federated learning with heterogeneous datacited by 9

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2023: 7; 2024: 25; 2025: 17; 2026: 6.

55 papers.

Sort:

By area:

  1. 2024FedTGP: Trainable Global Prototypes with Adaptive-Margin-Enhanced Contrastive Learning for Data and Model Heterogeneity in Federated Learningcited by 47
  2. 2023Deep Neural Collapse Is Provably Optimal for the Deep Unconstrained Features Modelcited by 43
  3. 2024Linguistic Collapse: Neural Collapse in (Large) Language Modelscited by 37
  4. 2024FedLLM-Bench: Realistic Benchmarks for Federated Learning of Large Language Modelscited by 30
  5. 2024Fair Federated Learning under Domain Skew with Local Consistency and Domain Diversitycited by 29
  6. 2025FedAWA: Adaptive Optimization of Aggregation Weights in Federated Learning Using Client Vectorscited by 19
  7. 2024Emerging Safety Attack and Defense in Federated Instruction Tuning of Large Language Modelscited by 18
  8. 2024An Upload-Efficient Scheme for Transferring Knowledge From a Server-Side Pre-trained Generator to Clients in Heterogeneous Federated Learningcited by 17
  9. 2024Neural Collapse versus Low-rank Bias: Is Deep Neural Collapse Really Optimal?cited by 17
  10. 2024Wide Neural Networks Trained with Weight Decay Provably Exhibit Neural Collapsecited by 16
  11. 2024FedImpro: Measuring and Improving Client Update in Federated Learningcited by 15
  12. 2023FedSOL: Stabilized Orthogonal Learning with Proximal Restrictions in Federated Learningcited by 14
38 more
  1. 2024Navigate Beyond Shortcuts: Debiased Learning through the Lens of Neural Collapsecited by 13
  2. 2025Geometric Knowledge-Guided Localized Global Distribution Alignment for Federated Learningcited by 12
  3. 2024FuseFL: One-Shot Federated Learning through the Lens of Causality with Progressive Model Fusioncited by 11
  4. 2024FedLoGe: Joint Local and Generic Federated Learning under Long-tailed Datacited by 10
  5. 2024Accelerating Heterogeneous Federated Learning with Closed-form Classifierscited by 10
  6. 2025Neural Collapse is Globally Optimal in Deep Regularized ResNets and Transformerscited by 9
  7. 2023PRIOR: Personalized Prior for Reactivating the Information Overlooked in Federated Learningcited by 8
  8. 2023FedFN: Feature Normalization for Alleviating Data Heterogeneity Problem in Federated Learningcited by 8
  9. 2024Improving Model Fusion by Training-time Neuron Alignment with Fixed Neuron Anchorscited by 7
  10. 2024Mamba-FSCIL: Dynamic Adaptation with Selective State Space Model for Few-Shot Class-Incremental Learningcited by 7
  11. 2025FedLWS: Federated Learning with Adaptive Layer-wise Weight Shrinkingcited by 7
  12. 2025Subgraph Federated Learning for Local Generalizationcited by 6
  13. 2023FediOS: Decoupling Orthogonal Subspaces for Personalization in Feature-skew Federated Learningcited by 5
  14. 2025You Are Your Own Best Teacher: Achieving Centralized-level Performance in Federated Learning under Heterogeneous and Long-tailed Datacited by 5
  15. 2023Pairwise Similarity Learning is SimPLEcited by 4
  16. 2024Covariances for Free: Exploiting Mean Distributions for Training-free Federated Learningcited by 3
  17. 2023Understanding Prompt Tuning for V-L Models Through the Lens of Neural Collapsecited by 2
  18. 2024FedDr+: Stabilizing Dot-regression with Global Feature Distillation for Federated Learningcited by 2
  19. 2024Tackling Feature-Classifier Mismatch in Federated Learning via Prompt-Driven Feature Transformationcited by 2
  20. 2024DualFed: Enjoying both Generalization and Personalization in Federated Learning via Hierachical Representationscited by 2
  21. 2025FedBM: Stealing Knowledge from Pre-trained Language Models for Heterogeneous Federated Learningcited by 2
  22. 2025Enhancing Visual Representation with Textual Semantics: Textual Semantics-Powered Prototypes for Heterogeneous Federated Learningcited by 2
  23. 2025Federated Adapter on Foundation Models: An Out-Of-Distribution Approachcited by 2
  24. 2025Neural Collapse-Inspired Multi-Label Federated Learning under Label-Distribution Skewcited by 2
  25. 2024Evaluating the Fairness of Neural Collapse in Medical Image Classificationcited by 1
  26. 2024Collapsed Language Models Promote Fairnesscited by 1
  27. 2024TPFL: A Trustworthy Personalized Federated Learning Framework via Subjective Logiccited by 1
  28. 2025FedORGP: Guiding Heterogeneous Federated Learning with Orthogonality Regularization on Global Prototypescited by 1
  29. 2025FedDEAP: Adaptive Dual-Prompt Tuning for Multi-Domain Federated Learningcited by 1
  30. 2025FedRE: A Representation Entanglement Framework for Model-Heterogeneous Federated Learningcited by 1
  31. 2026GFPL: Generative Federated Prototype Learning for Resource-Constrained and Data-Imbalanced Vision Taskcited by 1
  32. 2024Fixed Random Classifier Rearrangement for Continual Learningcited by 0
  33. 2024FedGuCci: Making Local Models More Connected in Landscape for Federated Learningcited by 0
  34. 2024Client2Vec: Improving Federated Learning by Distribution Shifts Aware Client Indexingcited by 0
  35. 2025Knowledge-Driven Federated Graph Learning on Model Heterogeneitycited by 0
  36. 2025Calibrating Biased Distribution in VFM-derived Latent Space via Cross-Domain Geometric Consistencycited by 0
  37. 2025Rethinking the Backbone in Class Imbalanced Federated Source Free Domain Adaptation: The Utility of Vision Foundation Modelscited by 0
  38. 2025From Coefficients to Directions: Rethinking Model Merging with Directional Alignmentcited by 0

50 most cited of 55 All 55 (JSON)