mlsanityData to 14 Oct 2026

FedTGP: Trainable Global Prototypes with Adaptive-Margin-Enhanced Contrastive Learning for Data and Model Heterogeneity in Federated Learning

arXiv 2401.03230 ↗First posted 6 Jan 2024cs.LG cs.CR cs.DCcited by 47

Not on the map. Why?

Abstract (arXiv)

Recently, Heterogeneous Federated Learning (HtFL) has attracted attention due to its ability to support heterogeneous models and data. To reduce the high communication cost of transmitting model parameters, a major challenge in HtFL, prototype-based HtFL methods are proposed to solely share class representatives, a.k.a, prototypes, among heterogeneous clients while maintaining the privacy of clients' models. However, these prototypes are naively aggregated into global prototypes on the server using weighted averaging, resulting in suboptimal global knowledge which negatively impacts the performance of clients. To overcome this challenge, we introduce a novel HtFL approach called FedTGP, which leverages our Adaptive-margin-enhanced Contrastive Learning (ACL) to learn Trainable Global Prototypes (TGP) on the server. By incorporating ACL, our approach enhances prototype separability while preserving semantic meaning. Extensive experiments with twelve heterogeneous models demonstrate that our FedTGP surpasses state-of-the-art methods by up to 9.08% in accuracy while maintaining the communication and privacy advantages of prototype-based HtFL. Our code is available at https://github.com/TsingZ0/FedTGP.

Key facts. Cited by 47 papers in the map (data through 14 Oct 2026). Builds on 45 of its 71 references that are in the map. First posted on arXiv 6 Jan 2024.

Builds on

45 of 71 references.

References · 45

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  3. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  4. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,795
  5. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,349
  6. 2014Going Deeper with Convolutionscited by 4,241
  7. 2019Advances and Open Problems in Federated Learningcited by 3,046
  8. 2015FaceNet: A Unified Embedding for Face Recognition and Clusteringcited by 2,849
  9. 2020Ensemble Distillation for Robust Model Fusion in Federated Learningcited by 682
  10. 2020Ditto: Fair and Robust Federated Learning Through Personalizationcited by 672
  11. 2017Deep Mutual Learningcited by 628
  12. 2020Personalized Federated Learning with Moreau Envelopescited by 618
33 more
  1. 2021Federated Learning on Non-IID Data Silos: An Experimental Studycited by 568
  2. 2020Federated Learning with Matched Averagingcited by 564
  3. 2021Exploiting Shared Representations for Personalized Federated Learningcited by 541
  4. 2019FedMD: Heterogenous Federated Learning via Model Distillationcited by 459
  5. 2021Data-Free Knowledge Distillation for Heterogeneous Federated Learningcited by 383
  6. 2017A Downsampled Variant of ImageNet as an Alternative to the CIFAR datasetscited by 377
  7. 2019A Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protectioncited by 374
  8. 2021FedProto: Federated Prototype Learning across Heterogeneous Clientscited by 352
  9. 2020HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clientscited by 337
  10. 2020Think Locally, Act Globally: Federated Learning with Local and Global Representationscited by 325
  11. 2018Communication-Efficient On-Device Machine Learning: Federated Distillation and Augmentation under Non-IID Private Datacited by 243
  12. 2021No Fear of Heterogeneity: Classifier Calibration for Federated Learning with Non-IID Datacited by 241
  13. 2021Personalized Federated Learning using Hypernetworkscited by 212
  14. 2021FjORD: Fair and Accurate Federated Learning under heterogeneous targets with Ordered Dropoutcited by 192
  15. 2021FedKD: Communication Efficient Federated Learning via Knowledge Distillationcited by 187
  16. 2022Fine-tuning Global Model via Data-Free Knowledge Distillation for Non-IID Federated Learningcited by 172
  17. 2020Attribute Prototype Network for Zero-Shot Learningcited by 160
  18. 2022Federated Learning from Pre-Trained Models: A Contrastive Learning Approachcited by 142
  19. 2018Robust Classification with Convolutional Prototype Learningcited by 123
  20. 2022FedALA: Adaptive Local Aggregation for Personalized Federated Learningcited by 122
  21. 2021Parameterized Knowledge Transfer for Personalized Federated Learningcited by 109
  22. 2022Layer-wised Model Aggregation for Personalized Federated Learningcited by 96
  23. 2023Multimodal Federated Learning via Contrastive Representation Ensemblecited by 86
  24. 2017Unsupervised Domain Adaptation with Similarity Learningcited by 73
  25. 2020Federated Mutual Learningcited by 71
  26. 2020Attract, Perturb, and Explore: Learning a Feature Alignment Network for Semi-supervised Domain Adaptationcited by 69
  27. 2021A Prototype-Oriented Framework for Unsupervised Domain Adaptationcited by 62
  28. 2023FedGH: Heterogeneous Federated Learning with Generalized Global Headercited by 56
  29. 2023No Fear of Classifier Biases: Neural Collapse Inspired Federated Learning with Synthetic and Fixed Classifiercited by 55
  30. 2021Scaling Law for Recommendation Models: Towards General-purpose User Representationscited by 26
  31. 2020AMC-Loss: Angular Margin Contrastive Loss for Improved Explainability in Image Classificationcited by 14
  32. 2018IamNN: Iterative and Adaptive Mobile Neural Network for Efficient Image Classificationcited by 12
  33. 2022A Closer Look at Branch Classifiers of Multi-exit Architecturescited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 9; 2025: 22; 2026: 16.

47 papers.

Sort:

By area:

  1. 2024FedLLM-Bench: Realistic Benchmarks for Federated Learning of Large Language Modelscited by 30
  2. 2024Cross-Modal Prototype based Multimodal Federated Learning under Severely Missing Modalitycited by 21
  3. 2024Federated Model Heterogeneous Matryoshka Representation Learningcited by 21
  4. 2024An Upload-Efficient Scheme for Transferring Knowledge From a Server-Side Pre-trained Generator to Clients in Heterogeneous Federated Learningcited by 17
  5. 2024OpenFGL: A Comprehensive Benchmark for Federated Graph Learningcited by 16
  6. 2026MM-OpenFGL: A Comprehensive Benchmark for Multimodal Federated Graph Learningcited by 11
  7. 2025FedSA: A Unified Representation Learning via Semantic Anchors for Prototype-based Federated Learningcited by 8
  8. 2025HtFLlib: A Comprehensive Heterogeneous Federated Learning Library and Benchmarkcited by 7
  9. 2025You Are Your Own Best Teacher: Achieving Centralized-level Performance in Federated Learning under Heterogeneous and Long-tailed Datacited by 5
  10. 2024FedLF: Adaptive Logit Adjustment and Feature Optimization in Federated Long-Tailed Learningcited by 4
  11. 2025Towards Harnessing the Collaborative Power of Large and Small Models for Domain Taskscited by 4
  12. 2024FedHPL: Efficient Heterogeneous Federated Learning with Prompt Tuning and Logit Distillationcited by 3
35 more
  1. 2026STAGE: Tackling Semantic Drift in Multimodal Federated Graph Learningcited by 3
  2. 2025Enhancing Visual Representation with Textual Semantics: Textual Semantics-Powered Prototypes for Heterogeneous Federated Learningcited by 2
  3. 2025Mosaic: Data-Free Knowledge Distillation via Mixture-of-Experts for Heterogeneous Distributed Environmentscited by 2
  4. 2026Toward Enhancing Representation Learning in Federated Multi-Task Settingscited by 2
  5. 2024Modality Alignment Meets Federated Broadcastingcited by 1
  6. 2025FedORGP: Guiding Heterogeneous Federated Learning with Orthogonality Regularization on Global Prototypescited by 1
  7. 2025FedDyMem: Efficient Federated Learning with Dynamic Memory and Memory-Reduce for Unsupervised Image Anomaly Detectioncited by 1
  8. 2025Federated Prototype Graph Learningcited by 1
  9. 2025Bridging Generalization Gap of Heterogeneous Federated Clients Using Generative Modelscited by 1
  10. 2025FedRE: A Representation Entanglement Framework for Model-Heterogeneous Federated Learningcited by 1
  11. 2026GFPL: Generative Federated Prototype Learning for Resource-Constrained and Data-Imbalanced Vision Taskcited by 1
  12. 2024Adaptive Guidance for Local Training in Heterogeneous Federated Learningcited by 0
  13. 2025Knowledge-Driven Federated Graph Learning on Model Heterogeneitycited by 0
  14. 2025Adaptive Prototype Knowledge Transfer for Federated Learning with Mixed Modalities and Heterogeneous Taskscited by 0
  15. 2025Provably Near-Optimal Federated Ensemble Distillation with Negligible Overheadcited by 0
  16. 2025FedMSGL: A Self-Expressive Hypergraph Based Federated Multi-View Learningcited by 0
  17. 2025Harmonizing Generalization and Personalization in Ring-topology Decentralized Federated Learningcited by 0
  18. 2025Heterogeneous Federated Learning with Prototype Alignment and Upscalingcited by 0
  19. 2025TinyProto: Communication-Efficient Federated Learning with Sparse Prototypes in Resource-Constrained Environmentscited by 0
  20. 2025Domain Borders Are There to Be Crossed With Federated Few-Shot Adaptationcited by 0
  21. 2025Dual-Distilled Heterogeneous Federated Learning with Adaptive Margins for Trainable Global Prototypescited by 0
  22. 2025Sentinel: Dynamic Knowledge Distillation for Personalized Federated Intrusion Detection in Heterogeneous IoT Networkscited by 0
  23. 2025Stragglers Can Contribute More: Uncertainty-Aware Distillation for Asynchronous Federated Learningcited by 0
  24. 2026Breaking the Prototype Bias Loop: Confidence-Aware Federated Contrastive Learning for Highly Imbalanced Clientscited by 0
  25. 2026FedEU: Evidential Uncertainty-Driven Federated Fine-Tuning of Vision Foundation Models for Remote Sensing Image Segmentationcited by 0
  26. 2026Domain-Skewed Federated Learning with Feature Decoupling and Calibrationcited by 0
  27. 2026FedDES: Graph-Based Dynamic Ensemble Selection for Personalized Federated Learningcited by 0
  28. 2026FedDBP: Enhancing Federated Prototype Learning with Dual-Branch Features and Personalized Global Fusioncited by 0
  29. 2026FedDAP: Domain-Aware Prototype Learning for Federated Learning under Domain Shiftcited by 0
  30. 2026From Coordinate Matching to Structural Alignment: Rethinking Prototype Alignment in Heterogeneous Federated Learningcited by 0
  31. 2026FedHPro: Federated Hyper-Prototype Learning via Gradient Matchingcited by 0
  32. 2026FedProIn: Mitigating Client Drift for Learnable Prototypes in Federated Medical Imagingcited by 0
  33. 2026FedTopo: Relation-Level Topology Sharing for Model-Heterogeneous Federated Learningcited by 0
  34. 2026Structural Entropy-Driven Graph Diffusion Generation for One-Shot Federated Graph Learningcited by 0
  35. 2026Prototype-guided Bilateral Alignment Multimodal Federated Learningcited by 0