# Tackling Data Heterogeneity in Federated Learning with Class Prototypes

arXiv: https://arxiv.org/abs/2212.02758 · HTML: https://mlsanity.com/paper/2212.02758/

## Key facts

- Cited by 67 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 43 references that are in the map.
- First posted on arXiv 6 Dec 2022.

## Builds on (most cited first, 28 of 28)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [Federated Optimization in Heterogeneous Networks](https://mlsanity.com/paper/1812.06127/) (2018)
- [Class-Balanced Loss Based on Effective Number of Samples](https://mlsanity.com/paper/1901.05555/) (2019)
- [Learning Imbalanced Datasets with Label-Distribution-Aware Margin Loss](https://mlsanity.com/paper/1906.07413/) (2019)
- [Decoupling Representation and Classifier for Long-Tailed Recognition](https://mlsanity.com/paper/1910.09217/) (2019)
- [Focal Loss for Dense Object Detection](https://mlsanity.com/paper/1708.02002/) (2017)
- [Ensemble Distillation for Robust Model Fusion in Federated Learning](https://mlsanity.com/paper/2006.07242/) (2020)
- [Ditto: Fair and Robust Federated Learning Through Personalization](https://mlsanity.com/paper/2012.04221/) (2020)
- [Local SGD Converges Fast and Communicates Little](https://mlsanity.com/paper/1805.09767/) (2018)
- [Federated Learning Based on Dynamic Regularization](https://mlsanity.com/paper/2111.04263/) (2021)
- [Federated Learning with Personalization Layers](https://mlsanity.com/paper/1912.00818/) (2019)
- [Exploiting Shared Representations for Personalized Federated Learning](https://mlsanity.com/paper/2102.07078/) (2021)
- [Towards Personalized Federated Learning](https://mlsanity.com/paper/2103.00710/) (2021)
- [BBN: Bilateral-Branch Network with Cumulative Learning for Long-Tailed Visual Recognition](https://mlsanity.com/paper/1912.02413/) (2019)
- [FedProto: Federated Prototype Learning across Heterogeneous Clients](https://mlsanity.com/paper/2105.00243/) (2021)
- [Additive Margin Softmax for Face Verification](https://mlsanity.com/paper/1801.05599/) (2018)
- [Parallel Restarted SGD with Faster Convergence and Less Communication: Demystifying Why Model Averaging Works for Deep Learning](https://mlsanity.com/paper/1807.06629/) (2018)
- [Long-tailed Recognition by Routing Diverse Distribution-Aware Experts](https://mlsanity.com/paper/2010.01809/) (2020)
- [NormFace: L2 Hypersphere Embedding for Face Verification](https://mlsanity.com/paper/1704.06369/) (2017)
- [FedBABU: Towards Enhanced Representation for Federated Image Classification](https://mlsanity.com/paper/2106.06042/) (2021)
- [Cost Sensitive Learning of Deep Feature Representations from Imbalanced Data](https://mlsanity.com/paper/1508.03422/) (2015)
- [Targeted Supervised Contrastive Learning for Long-Tailed Recognition](https://mlsanity.com/paper/2111.13998/) (2021)
- [Federated Learning on Heterogeneous and Long-Tailed Data via Classifier Re-Training with Federated Features](https://mlsanity.com/paper/2204.13399/) (2022)
- [Deep Learning using Rectified Linear Units (ReLU)](https://mlsanity.com/paper/1803.08375/) (2018)

## Built on by (most cited first, 30 of 67)

- [No Fear of Classifier Biases: Neural Collapse Inspired Federated Learning with Synthetic and Fixed Classifier](https://mlsanity.com/paper/2303.10058/) (2023)
- [Taming Cross-Domain Representation Variance in Federated Prototype Learning with Heterogeneous Data Domains](https://mlsanity.com/paper/2403.09048/) (2024)
- [PILoRA: Prototype Guided Incremental LoRA for Federated Class-Incremental Learning](https://mlsanity.com/paper/2401.02094/) (2024)
- [Multimodal Federated Learning with Missing Modality via Prototype Mask and Contrast](https://mlsanity.com/paper/2312.13508/) (2023)
- [Multi-level Personalized Federated Learning on Heterogeneous and Long-Tailed Data](https://mlsanity.com/paper/2405.06413/) (2024)
- [Non-IID data in Federated Learning: A Survey with Taxonomy, Metrics, Methods, Frameworks and Future Directions](https://mlsanity.com/paper/2411.12377/) (2024)
- [Federated Learning with Extremely Noisy Clients via Negative Distillation](https://mlsanity.com/paper/2312.12703/) (2023)
- [FuseFL: One-Shot Federated Learning through the Lens of Causality with Progressive Model Fusion](https://mlsanity.com/paper/2410.20380/) (2024)
- [FedLoGe: Joint Local and Generic Federated Learning under Long-tailed Data](https://mlsanity.com/paper/2401.08977/) (2024)
- [FedSA: A Unified Representation Learning via Semantic Anchors for Prototype-based Federated Learning](https://mlsanity.com/paper/2501.05496/) (2025)
- [Improving Model Fusion by Training-time Neuron Alignment with Fixed Neuron Anchors](https://mlsanity.com/paper/2402.01342/) (2024)
- [Physics-Driven Spectrum-Consistent Federated Learning for Palmprint Verification](https://mlsanity.com/paper/2308.00451/) (2023)
- [Overcome Modal Bias in Multi-modal Federated Learning via Balanced Modality Selection](https://mlsanity.com/paper/2401.00403/) (2023)
- [Enhancing Federated Domain Adaptation with Multi-Domain Prototype-Based Federated Fine-Tuning](https://mlsanity.com/paper/2410.07738/) (2024)
- [FediOS: Decoupling Orthogonal Subspaces for Personalization in Feature-skew Federated Learning](https://mlsanity.com/paper/2311.18559/) (2023)
- [FedRFQ: Prototype-Based Federated Learning with Reduced Redundancy, Minimal Failure, and Enhanced Quality](https://mlsanity.com/paper/2401.07558/) (2024)
- [FedCCL: Federated Dual-Clustered Feature Contrast Under Domain Heterogeneity](https://mlsanity.com/paper/2404.09259/) (2024)
- [You Are Your Own Best Teacher: Achieving Centralized-level Performance in Federated Learning under Heterogeneous and Long-tailed Data](https://mlsanity.com/paper/2503.06916/) (2025)
- [Contribution Evaluation of Heterogeneous Participants in Federated Learning via Prototypical Representations](https://mlsanity.com/paper/2407.02073/) (2024)
- [Federated Client-tailored Adapter for Medical Image Segmentation](https://mlsanity.com/paper/2504.18020/) (2025)
- [An Empirical Study of Federated Prompt Learning for Vision Language Model](https://mlsanity.com/paper/2505.23024/) (2025)
- [FedSC: Federated Learning with Semantic-Aware Collaboration](https://mlsanity.com/paper/2506.21012/) (2025)
- [FedDifRC: Unlocking the Potential of Text-to-Image Diffusion Models in Heterogeneous Federated Learning](https://mlsanity.com/paper/2507.06482/) (2025)
- [Gradient-Congruity Guided Federated Sparse Training](https://mlsanity.com/paper/2405.01189/) (2024)
- [Maverick-Aware Shapley Valuation for Client Selection in Federated Learning](https://mlsanity.com/paper/2405.12590/) (2024)
- [Personalized federated learning based on feature fusion](https://mlsanity.com/paper/2406.16583/) (2024)
- [FedAli: Personalized Federated Learning Alignment with Prototype Layers for Generalized Mobile Services](https://mlsanity.com/paper/2411.10595/) (2024)
- [FedBM: Stealing Knowledge from Pre-trained Language Models for Heterogeneous Federated Learning](https://mlsanity.com/paper/2502.16832/) (2025)
- [Enhancing Visual Representation with Textual Semantics: Textual Semantics-Powered Prototypes for Heterogeneous Federated Learning](https://mlsanity.com/paper/2503.13543/) (2025)
- [Mosaic: Data-Free Knowledge Distillation via Mixture-of-Experts for Heterogeneous Distributed Environments](https://mlsanity.com/paper/2505.19699/) (2025)

## Abstract (arXiv)

Data heterogeneity across clients in federated learning (FL) settings is a widely acknowledged challenge. In response, personalized federated learning (PFL) emerged as a framework to curate local models for clients' tasks. In PFL, a common strategy is to develop local and global models jointly - the global model (for generalization) informs the local models, and the local models (for personalization) are aggregated to update the global model. A key observation is that if we can improve the generalization ability of local models, then we can improve the generalization of global models, which in turn builds better personalized models. In this work, we consider class imbalance, an overlooked type of data heterogeneity, in the classification setting. We propose FedNH, a novel method that improves the local models' performance for both personalization and generalization by combining the uniformity and semantics of class prototypes. FedNH initially distributes class prototypes uniformly in the latent space and smoothly infuses the class semantics into class prototypes. We show that imposing uniformity helps to combat prototype collapse while infusing class semantics improves local models. Extensive experiments were conducted on popular classification datasets under the cross-device setting. Our results demonstrate the effectiveness and stability of our method over recent works.