mlsanityData to 14 Oct 2026

FedCiR: Client-Invariant Representation Learning for Federated Non-IID Features

arXiv 2308.15786 ↗First posted 30 Aug 2023cs.LG cs.AIcited by 12

Not on the map. Why?

Abstract (arXiv)

Federated learning (FL) is a distributed learning paradigm that maximizes the potential of data-driven models for edge devices without sharing their raw data. However, devices often have non-independent and identically distributed (non-IID) data, meaning their local data distributions can vary significantly. The heterogeneity in input data distributions across devices, commonly referred to as the feature shift problem, can adversely impact the training convergence and accuracy of the global model. To analyze the intrinsic causes of the feature shift problem, we develop a generalization error bound in FL, which motivates us to propose FedCiR, a client-invariant representation learning framework that enables clients to extract informative and client-invariant features. Specifically, we improve the mutual information term between representations and labels to encourage representations to carry essential classification knowledge, and diminish the mutual information term between the client set and representations conditioned on labels to promote representations of clients to be client-invariant. We further incorporate two regularizers into the FL framework to bound the mutual information terms with an approximate global representation distribution to compensate for the absence of the ground-truth global representation distribution, thus achieving informative and client-invariant feature extraction. To achieve global representation distribution approximation, we propose a data-free mechanism performed by the server without compromising privacy. Extensive experiments demonstrate the effectiveness of our approach in achieving client-invariant representation learning and solving the data heterogeneity issue.

Key facts. Cited by 12 papers in the map (data through 14 Oct 2026). Builds on 39 of its 59 references that are in the map. First posted on arXiv 30 Aug 2023.

Builds on

39 of 59 references.

References · 39

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  3. 2019Advances and Open Problems in Federated Learningcited by 3,046
  4. 2018Federated Optimization in Heterogeneous Networkscited by 2,987
  5. 2020WILDS: A Benchmark of in-the-Wild Distribution Shiftscited by 1,375
  6. 2018Moment Matching for Multi-Source Domain Adaptationcited by 1,331
  7. 2016Deep CORAL: Correlation Alignment for Deep Domain Adaptationcited by 1,200
  8. 2018Federated Learning with Non-IID Datacited by 1,108
  9. 2019Deep Leakage from Gradientscited by 1,021
  10. 2017Deeper, Broader and Artier Domain Generalizationcited by 986
  11. 2021Generalizing to Unseen Domains: A Survey on Domain Generalizationcited by 766
  12. 2021Model-Contrastive Federated Learningcited by 766
27 more
  1. 2021FedBN: Federated Learning on Non-IID Features via Local Batch Normalizationcited by 571
  2. 2021Federated Learning on Non-IID Data Silos: An Experimental Studycited by 568
  3. 2021Federated Learning Based on Dynamic Regularizationcited by 562
  4. 2021Data-Free Knowledge Distillation for Heterogeneous Federated Learningcited by 383
  5. 2021FedProto: Federated Prototype Learning across Heterogeneous Clientscited by 352
  6. 2021Federated Learning on Non-IID Data: A Surveycited by 346
  7. 2019Tighter Theory for Local SGD on Identical and Heterogeneous Datacited by 266
  8. 2022Safety-Enhanced Autonomous Driving Using Interpretable Sensor Fusion Transformercited by 212
  9. 2022Fine-tuning Global Model via Data-Free Knowledge Distillation for Non-IID Federated Learningcited by 172
  10. 2019Federated Adversarial Domain Adaptationcited by 140
  11. 2019Generalizing to unseen domains via distribution matchingcited by 110
  12. 2021FedMix: Approximation of Mixup under Mean Augmented Federated Learningcited by 108
  13. 2018Visual Domain Adaptation with Manifold Embedded Distribution Alignmentcited by 91
  14. 2021FedProc: Prototypical Contrastive Federated Learning on Non-IID datacited by 81
  15. 2021The iWildCam 2021 Competition Datasetcited by 77
  16. 2019Correlation-aware Adversarial Domain Adaptation and Generalizationcited by 65
  17. 2021HarmoFL: Harmonizing Local and Global Drifts in Federated Learning on Heterogeneous Medical Imagescited by 64
  18. 2022Virtual Homogeneity Learning: Defending against Data Heterogeneity in Federated Learningcited by 52
  19. 2022FedFA: Federated Learning with Feature Anchors to Align Features and Classifiers for Heterogeneous Datacited by 50
  20. 2021Federated Learning with Domain Generalizationcited by 41
  21. 2022Federated Learning with GAN-based Data Synthesis for Non-IID Clientscited by 25
  22. 2021FedDPGAN: Federated Differentially Private Generative Adversarial Networks Framework for the Detection of COVID-19 Pneumoniacited by 22
  23. 2023A Survey of What to Share in Federated Learning: Perspectives on Model Utility, Privacy Leakage, and Communication Efficiencycited by 21
  24. 2021Federated Learning for COVID-19 Detection with Generative Adversarial Networks in Edge Cloud Computingcited by 18
  25. 2023Feature Matching Data Synthesis for Non-IID Federated Learningcited by 13
  26. 2020Federated Learning for Task and Resource Allocation in Wireless High Altitude Balloon Networkscited by 11
  27. 2021Relay-Assisted Cooperative Federated Learningcited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 3; 2025: 5; 2026: 4.

12 papers.

  1. 2024Non-IID data in Federated Learning: A Survey with Taxonomy, Metrics, Methods, Frameworks and Future Directionscited by 13
  2. 2024Federated Prompt-based Decision Transformer for Customized VR Services in Mobile Edge Computing Systemcited by 2
  3. 2025FedPall: Prototype-based Adversarial and Collaborative Learning for Federated Learning with Feature Driftcited by 2
  4. 2024LoLaFL: Low-Latency Federated Learning via Forward-only Propagationcited by 1
  5. 2025FedDyMem: Efficient Federated Learning with Dynamic Memory and Memory-Reduce for Unsupervised Image Anomaly Detectioncited by 1
  6. 2025FedSDWC: Federated Synergistic Dual-Representation Weak Causal Learning for OODcited by 0
  7. 2025Closing the Generalization Gap in Parameter-efficient Federated Edge Learningcited by 0
  8. 2025ALIGN-FL: Architecture-independent Learning through Invariant Generative component sharing in Federated Learningcited by 0
  9. 2026FedCova: Robust Federated Covariance Learning Against Noisy Labelscited by 0
  10. 2026GAPSL: A Gradient-Aligned Parallel Split Learning over Data-Heterogeneous Edge Computing Systemscited by 0
  11. 2026FedCausal-Dyn: A Causal-Dynamic Paradigm for Federated Learning under Dynamic Feature Driftcited by 0
  12. 2026When Is Shallow Enough? Adaptive Split Federated Learning with Client-Specific Sufficiency Estimationcited by 0