mlsanityData to 14 Oct 2026

FedFN: Feature Normalization for Alleviating Data Heterogeneity Problem in Federated Learning

arXiv 2311.13267 ↗First posted 22 Nov 2023cs.LG cs.AI cs.CVcited by 8

Not on the map. Why?

Abstract (arXiv)

Federated Learning (FL) is a collaborative method for training models while preserving data privacy in decentralized settings. However, FL encounters challenges related to data heterogeneity, which can result in performance degradation. In our study, we observe that as data heterogeneity increases, feature representation in the FedAVG model deteriorates more significantly compared to classifier weight. Additionally, we observe that as data heterogeneity increases, the gap between higher feature norms for observed classes, obtained from local models, and feature norms of unobserved classes widens, in contrast to the behavior of classifier weight norms. This widening gap extends to encompass the feature norm disparities between local and the global models. To address these issues, we introduce Federated Averaging with Feature Normalization Update (FedFN), a straightforward learning method. We demonstrate the superior performance of FedFN through extensive experiments, even when applied to pretrained ResNet18. Subsequently, we confirm the applicability of FedFN to foundation models.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 36 of its 40 references that are in the map. First posted on arXiv 22 Nov 2023.

Builds on

36 of 40 references.

References · 36

  1. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  3. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,845
  4. 2018Federated Optimization in Heterogeneous Networkscited by 2,987
  5. 2020Supervised Contrastive Learningcited by 2,829
  6. 2016Identity Mappings in Deep Residual Networkscited by 1,965
  7. 2019On the Convergence of FedAvg on Non-IID Datacited by 1,229
  8. 2020Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimizationcited by 906
  9. 2019Decoupling Representation and Classifier for Long-Tailed Recognitioncited by 809
  10. 2021Model-Contrastive Federated Learningcited by 766
  11. 2019Large Scale Incremental Learningcited by 732
  12. 2020Prevalence of Neural Collapse during the terminal phase of deep learning trainingcited by 719
24 more
  1. 2020Ensemble Distillation for Robust Model Fusion in Federated Learningcited by 682
  2. 2020Ditto: Fair and Robust Federated Learning Through Personalizationcited by 672
  3. 2020Federated Learning with Matched Averagingcited by 564
  4. 2019Federated Learning with Personalization Layerscited by 541
  5. 2021Exploiting Shared Representations for Personalized Federated Learningcited by 541
  6. 2021FedProto: Federated Prototype Learning across Heterogeneous Clientscited by 352
  7. 2019Maintaining Discrimination and Fairness in Class Incremental Learningcited by 283
  8. 2018Additive Margin Softmax for Face Verificationcited by 278
  9. 2021No Fear of Heterogeneity: Classifier Calibration for Federated Learning with Non-IID Datacited by 241
  10. 2021FedBABU: Towards Enhanced Representation for Federated Image Classificationcited by 182
  11. 2022Federated Learning from Pre-Trained Models: A Contrastive Learning Approachcited by 142
  12. 2023Personalized Federated Learning with Feature Alignment and Classifier Collaborationcited by 97
  13. 2019Hyperspherical Prototype Networkscited by 86
  14. 2022On the Importance and Applicability of Pre-Training for Federated Learningcited by 71
  15. 2022Where to Begin? On the Impact of Pre-Training and Initialization in Federated Learningcited by 71
  16. 2023FedExP: Speeding Up Federated Averaging via Extrapolationcited by 69
  17. 2023No Fear of Classifier Biases: Neural Collapse Inspired Federated Learning with Synthetic and Fixed Classifiercited by 55
  18. 2023Federated Foundation Models: Privacy-Preserving and Collaborative Learning for Large Modelscited by 50
  19. 2022Towards Understanding and Mitigating Dimensional Collapse in Heterogeneous Federated Learningcited by 46
  20. 2021Fed2: Feature-Aligned Federated Learningcited by 42
  21. 2022Federated Learning with Position-Aware Neuronscited by 24
  22. 2022SphereFed: Hyperspherical Federated Learningcited by 19
  23. 2020The Devil is the Classifier: Investigating Long Tail Relation Classification with Decoupling Analysiscited by 10
  24. 2017DeepVisage: Making face recognition simple yet with powerful generalization skillscited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 3; 2025: 4; 2026: 1.

8 papers.

  1. 2024Covariances for Free: Exploiting Mean Distributions for Training-free Federated Learningcited by 4
  2. 2024Federated Learning for Time-Series Healthcare Sensing with Incomplete Modalitiescited by 3
  3. 2024FedDr+: Stabilizing Dot-regression with Global Feature Distillation for Federated Learningcited by 2
  4. 2025Foundational Models and Federated Learning: Survey, Taxonomy, Challenges and Practical Insightscited by 1
  5. 2025FedRE: A Representation Entanglement Framework for Model-Heterogeneous Federated Learningcited by 1
  6. 2025Provably Near-Optimal Federated Ensemble Distillation with Negligible Overheadcited by 0
  7. 2025Hybrid Batch Normalisation: Resolving the Dilemma of Batch Normalisation in Federated Learningcited by 0
  8. 2026Global Federated Learning Strategies for Building Efficient Personalized Modelscited by 0