Data to 7 Oct 2026

FedProto: Federated Prototype Learning across Heterogeneous Clients

arXiv 2105.00243 ↗First posted 1 May 2021cs.LG cs.DCcited by 349

Not on the map. Why?

Abstract (arXiv)

Heterogeneity across clients in federated learning (FL) usually hinders the optimization convergence and generalization performance when the aggregation of clients' knowledge occurs in the gradient space. For example, clients may differ in terms of data distribution, network latency, input/output space, and/or model architecture, which can easily lead to the misalignment of their local gradients. To improve the tolerance to heterogeneity, we propose a novel federated prototype learning (FedProto) framework in which the clients and server communicate the abstract class prototypes instead of the gradients. FedProto aggregates the local prototypes collected from different clients, and then sends the global prototypes back to all clients to regularize the training of local models. The training on each client aims to minimize the classification error on the local data while keeping the resulting local prototypes sufficiently close to the corresponding global ones. Moreover, we provide a theoretical analysis to the convergence rate of FedProto under non-convex objectives. In experiments, we propose a benchmark setting tailored for heterogeneous FL, with FedProto outperforming several recent FL approaches on multiple datasets.

Key facts. Cited by 349 papers in the map (data through 7 Oct 2026). Builds on 37 of its 55 references that are in the map. First posted on arXiv 1 May 2021.

Builds on

37 of 55 references.

References · 37

2015Deep Residual Learning for Image Recognitioncited by 51,809
2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
2019Advances and Open Problems in Federated Learningcited by 3,021
2017Prototypical Networks for Few-shot Learningcited by 3,001
2018Federated Optimization in Heterogeneous Networkscited by 2,971
2014Two-Stream Convolutional Networks for Action Recognition in Videoscited by 1,067
2019Deep Leakage from Gradientscited by 1,004
2020Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimizationcited by 897
2018LEAF: A Benchmark for Federated Settingscited by 780
2021Model-Contrastive Federated Learningcited by 762
2020Ensemble Distillation for Robust Model Fusion in Federated Learningcited by 678
2020The Future of Digital Health with Federated Learningcited by 625

25 more

2020Federated Learning with Matched Averagingcited by 564
2021Exploiting Shared Representations for Personalized Federated Learningcited by 537
2019Federated Learning with Personalization Layerscited by 533
2020An Efficient Framework for Clustered Federated Learningcited by 505
2019Clustered Federated Learning: Model-Agnostic Distributed Multi-Task Optimization under Privacy Constraintscited by 499
2021Towards Personalized Federated Learningcited by 462
2019FedMD: Heterogenous Federated Learning via Model Distillationcited by 454
2020Adaptive Personalized Federated Learningcited by 353
2020Think Locally, Act Globally: Federated Learning with Local and Global Representationscited by 322
2020Three Approaches for Personalization with Applications to Federated Learningcited by 321
2020FedML: A Research Library and Benchmark for Federated Machine Learningcited by 298
2021Graph Self-Supervised Learning: A Surveycited by 289
2018Communication-Efficient On-Device Machine Learning: Federated Distillation and Augmentation under Non-IID Private Datacited by 242
2021Anomaly Detection on Attributed Networks via Contrastive Self-Supervised Learningcited by 175
2020A Universal Representation Transformer Layer for Few-Shot Image Classificationcited by 65
2015Aggregating Deep Convolutional Features for Image Retrievalcited by 63
2015Towards Universal Paraphrastic Sentence Embeddingscited by 59
2020Interpretable and Efficient Heterogeneous Graph Convolutional Networkcited by 48
2021Dynamic Network Embedding Surveycited by 44
2020From Federated Learning to Federated Neural Architecture Search: A Surveycited by 42
2019Real-World Image Datasets for Federated Learningcited by 30
2020Towards Non-I.I.D. and Invisible Data with FedNAS: Federated Deep Learning via Neural Architecture Searchcited by 30
2020Selecting Relevant Features from a Multi-domain Representation for Few-shot Classificationcited by 11
2020Differentially-private Federated Neural Architecture Searchcited by 9
2020Robust Federated Recommendation Systemcited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 17; 2023: 62; 2024: 100; 2025: 95; 2026: 74.

349 papers.

Sort:

By area:

2023Heterogeneous Federated Learning: State-of-the-art and Research Challengescited by 160
2023Federated Learning for Generalization, Robustness, Fairness: A Survey and Benchmarkcited by 97
2022Rethinking and Scaling Up Graph Contrastive Learning: An Extremely Efficient Approach with Group Discriminationcited by 67
2022Tackling Data Heterogeneity in Federated Learning with Class Prototypescited by 66
2021Personalized Federated Learning through Local Memorizationcited by 57
2022Beyond Smoothing: Unsupervised Graph Representation Learning with Edge Heterophily Discriminatingcited by 57
2023FedGH: Heterogeneous Federated Learning with Generalized Global Headercited by 56
2023No Fear of Classifier Biases: Neural Collapse Inspired Federated Learning with Synthetic and Fixed Classifiercited by 55
2023Efficient Model Personalization in Federated Learning via Client-Specific Prompt Generationcited by 54
2023FedFed: Feature Distillation against Data Heterogeneity in Federated Learningcited by 54
2022FedFA: Federated Learning with Feature Anchors to Align Features and Classifiers for Heterogeneous Datacited by 49
2023FedSpeed: Larger Local Interval, Less Communication Round, and Higher Generalization Accuracycited by 49

38 more

2022GOOD-D: On Unsupervised Graph Out-Of-Distribution Detectioncited by 47
2024FedTGP: Trainable Global Prototypes with Adaptive-Margin-Enhanced Contrastive Learning for Data and Model Heterogeneity in Federated Learningcited by 47
2023pFedLoRA: Model-Heterogeneous Personalized Federated Learning with LoRA Tuningcited by 38
2023Towards Personalized Federated Learning via Heterogeneous Model Reassemblycited by 31
2022Architecture Agnostic Federated Learning for Neural Networkscited by 30
2025HiDe-LLaVA: Hierarchical Decoupling for Continual Instruction Tuning of Multimodal Large Language Modelcited by 29
2024Fair Federated Learning under Domain Skew with Local Consistency and Domain Diversitycited by 28
2022FedFM: Anchor-based Feature Matching for Data Heterogeneity in Federated Learningcited by 27
2024Taming Cross-Domain Representation Variance in Federated Prototype Learning with Heterogeneous Data Domainscited by 27
2023Learning Strong Graph Neural Networks with Weak Informationcited by 25
2024FedAC: An Adaptive Clustered Federated Learning Framework for Heterogeneous Datacited by 25
2022FRAug: Tackling Federated Learning with Non-IID Features via Representation Augmentationcited by 24
2023Bold but Cautious: Unlocking the Potential of Personalized Federated Learning through Cautiously Aggressive Collaborationcited by 23
2023DFRD: Data-Free Robustness Distillation for Heterogeneous Federated Learningcited by 23
2024Closed-form merging of parameter-efficient modules for Federated Continual Learningcited by 22
2023Understanding How Consistency Works in Federated Learning via Stage-wise Relaxed Initializationcited by 21
2023A Survey of What to Share in Federated Learning: Perspectives on Model Utility, Privacy Leakage, and Communication Efficiencycited by 21
2023FedPerfix: Towards Partial Model Personalization of Vision Transformers in Federated Learningcited by 21
2024Cross-Modal Prototype based Multimodal Federated Learning under Severely Missing Modalitycited by 21
2024pFedMoE: Data-Level Personalization with Mixture of Experts for Model-Heterogeneous Personalized Federated Learningcited by 21
2024Overcoming Data and Model Heterogeneities in Decentralized Federated Learning via Synthetic Anchorscited by 21
2024Federated Model Heterogeneous Matryoshka Representation Learningcited by 21
2022CalFAT: Calibrated Federated Adversarial Training with Label Skewnesscited by 20
2023MP-FedCL: Multiprototype Federated Contrastive Learning for Edge Intelligencecited by 20
2023Dual Personalization on Federated Recommendationcited by 19
2024PILoRA: Prototype Guided Incremental LoRA for Federated Class-Incremental Learningcited by 19
2025FedAWA: Adaptive Optimization of Aggregation Weights in Federated Learning Using Client Vectorscited by 19
2023GPFL: Simultaneously Learning Global and Personalized Feature Information for Personalized Federated Learningcited by 18
2023Multimodal Federated Learning with Missing Modality via Prototype Mask and Contrastcited by 17
2024FedMM: Federated Multi-Modal Learning with Modality Heterogeneity in Computational Pathologycited by 17
2024An Upload-Efficient Scheme for Transferring Knowledge From a Server-Side Pre-trained Generator to Clients in Heterogeneous Federated Learningcited by 17
2022Federated Self-supervised Learning for Heterogeneous Clientscited by 16
2023A Simple Data Augmentation for Feature Distribution Skewed Federated Learningcited by 16
2023PFLlib: A Beginner-Friendly and Comprehensive Personalized Federated Learning Library and Benchmarkcited by 16
2024COMAE: COMprehensive Attribute Exploration for Zero-shot Hashingcited by 16
2024FedBiP: Heterogeneous One-Shot Federated Learning with Personalized Latent Diffusion Modelscited by 16
2024Personalized Adapter for Large Meteorology Model on Devices: Towards Weather Foundation Modelscited by 15
2025LCFed: An Efficient Clustered Federated Learning Framework for Heterogeneous Datacited by 15

50 most cited of 349 All 349 (JSON)