mlsanityData to 7 Oct 2026

Client2Vec: Improving Federated Learning by Distribution Shifts Aware Client Indexing

arXiv 2405.16233 ↗First posted 25 May 2024cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Federated Learning (FL) is a privacy-preserving distributed machine learning paradigm. Nonetheless, the substantial distribution shifts among clients pose a considerable challenge to the performance of current FL algorithms. To mitigate this challenge, various methods have been proposed to enhance the FL training process. This paper endeavors to tackle the issue of data heterogeneity from another perspective -- by improving FL algorithms prior to the actual training stage. Specifically, we introduce the Client2Vec mechanism, which generates a unique client index for each client before the commencement of FL training. Subsequently, we leverage the generated client index to enhance the subsequent FL training process. To demonstrate the effectiveness of the proposed Client2Vec method, we conduct three case studies that assess the impact of the client index on the FL training process. These case studies encompass enhanced client sampling, model aggregation, and local training. Extensive experiments conducted on diverse datasets and model architectures show the efficacy of Client2Vec across all three case studies. Our code is avaliable at urlhttps://github.com/LINs-lab/client2vec.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 56 of its 70 references that are in the map. First posted on arXiv 25 May 2024.

Builds on

56 of 70 references.

References · 56

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  3. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  4. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  5. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,346
  6. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
  7. 2015Domain-Adversarial Training of Neural Networkscited by 3,023
  8. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  9. 2018Moment Matching for Multi-Source Domain Adaptationcited by 1,327
  10. 2017Adversarial Discriminative Domain Adaptationcited by 1,309
  11. 2018Federated Learning with Non-IID Datacited by 1,099
  12. 2019Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classificationcited by 836
38 more
  1. 2018LEAF: A Benchmark for Federated Settingscited by 780
  2. 2021Model-Contrastive Federated Learningcited by 762
  3. 2020Ensemble Distillation for Robust Model Fusion in Federated Learningcited by 678
  4. 2020Federated Learning with Matched Averagingcited by 564
  5. 2021Federated Learning Based on Dynamic Regularizationcited by 555
  6. 2019Bayesian Nonparametric Federated Learning of Neural Networkscited by 385
  7. 2021FedProto: Federated Prototype Learning across Heterogeneous Clientscited by 349
  8. 2021Escaping the Big Data Paradigm with Compact Transformerscited by 261
  9. 2018Communication-Efficient On-Device Machine Learning: Federated Distillation and Augmentation under Non-IID Private Datacited by 242
  10. 2021No Fear of Heterogeneity: Classifier Calibration for Federated Learning with Non-IID Datacited by 241
  11. 2018Don't Use Large Mini-Batches, Use Local SGDcited by 201
  12. 2020Mime: Mimicking Centralized Stochastic Algorithms in Federated Learningcited by 152
  13. 2019Federated Adversarial Domain Adaptationcited by 140
  14. 2021Local Learning Matters: Rethinking Data Heterogeneity in Federated Learningcited by 114
  15. 2021FedMix: Approximation of Mixup under Mean Augmented Federated Learningcited by 108
  16. 2022Federated Learning with Label Distribution Skew via Logits Calibrationcited by 108
  17. 2021Distributionally Robust Federated Averagingcited by 104
  18. 2020Robust Federated Learning: The Case of Affine Distribution Shiftscited by 102
  19. 2021Clustered Sampling: Low-Variance and Improved Representativity for Clients Selection in Federated Learningcited by 93
  20. 2019Cronus: Robust and Heterogeneous Collaborative Learning with Black-Box Knowledge Transfercited by 91
  21. 2021Preservation of the Global Knowledge by Not-True Distillation in Federated Learningcited by 87
  22. 2020Continuously Indexed Domain Adaptationcited by 74
  23. 2021Tackling System and Statistical Heterogeneity for Federated Learning with Adaptive Client Samplingcited by 66
  24. 2019Astraea: Self-balancing Federated Learning for Improving Classification Accuracy of Mobile Deep Learning Applicationscited by 64
  25. 2021Towards Fair Federated Learning with Zero-Shot Data Augmentationcited by 64
  26. 2020XOR Mixup: Privacy-Preserving Data Augmentation for One-Shot Federated Learningcited by 58
  27. 2023No Fear of Classifier Biases: Neural Collapse Inspired Federated Learning with Synthetic and Fixed Classifiercited by 55
  28. 2023FedFed: Feature Distillation against Data Heterogeneity in Federated Learningcited by 54
  29. 2019G-PATE: Scalable Differentially Private Data Generator via Private Aggregation of Teacher Discriminatorscited by 53
  30. 2022Virtual Homogeneity Learning: Defending against Data Heterogeneity in Federated Learningcited by 52
  31. 2022FedFA: Federated Learning with Feature Anchors to Align Features and Classifiers for Heterogeneous Datacited by 49
  32. 2022Towards Understanding and Mitigating Dimensional Collapse in Heterogeneous Federated Learningcited by 45
  33. 2022Test-Time Robust Personalization for Federated Learningcited by 39
  34. 2022FedBR: Improving Federated Learning on Heterogeneous Data via Local Learning Bias Reductioncited by 35
  35. 2022Federated Learning under Distributed Concept Driftcited by 35
  36. 2023On the Convergence of Federated Averaging with Cyclic Client Participationcited by 31
  37. 2022DELTA: Diverse Client Sampling for Fasting Federated Learningcited by 24
  38. 2021Towards Federated Learning on Time-Evolving Heterogeneous Datacited by 23

50 most cited of 56 All 56 (JSON)

Built on by

Not cited yet.