mlsanityData to 14 Oct 2026

Federated Learning under Covariate Shifts with Generalization Guarantees

arXiv 2306.05325 ↗First posted 8 Jun 2023cs.LGcited by 6

Not on the map. Why?

Abstract (arXiv)

This paper addresses intra-client and inter-client covariate shifts in federated learning (FL) with a focus on the overall generalization performance. To handle covariate shifts, we formulate a new global model training paradigm and propose Federated Importance-Weighted Empirical Risk Minimization (FTW-ERM) along with improving density ratio matching methods without requiring perfect knowledge of the supremum over true ratios. We also propose the communication-efficient variant FITW-ERM with the same level of privacy guarantees as those of classical ERM in FL. We theoretically show that FTW-ERM achieves smaller generalization error than classical ERM under certain settings. Experimental results demonstrate the superiority of FTW-ERM over existing FL baselines in challenging imbalanced federated settings in terms of data distribution shifts across clients.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 47 of its 104 references that are in the map. First posted on arXiv 8 Jun 2023.

Builds on

47 of 104 references.

References · 47

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  3. 2022Generative Adversarial Networkscited by 5,627
  4. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,349
  5. 2019Advances and Open Problems in Federated Learningcited by 3,046
  6. 2015Domain-Adversarial Training of Neural Networkscited by 3,044
  7. 2019Invariant Risk Minimizationcited by 1,854
  8. 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,538
  9. 2020WILDS: A Benchmark of in-the-Wild Distribution Shiftscited by 1,375
  10. 2019Deep Leakage from Gradientscited by 1,021
  11. 2020Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimizationcited by 906
  12. 2018LEAF: A Benchmark for Federated Settingscited by 792
35 more
  1. 2017Federated Multi-Task Learningcited by 672
  2. 2020Ditto: Fair and Robust Federated Learning Through Personalizationcited by 672
  3. 2018Deep Visual Domain Adaptation: A Surveycited by 635
  4. 2021FedBN: Federated Learning on Non-IID Features via Local Batch Normalizationcited by 571
  5. 2019Agnostic Federated Learningcited by 456
  6. 2020Three Approaches for Personalization with Applications to Federated Learningcited by 325
  7. 2021A Field Guide to Federated Optimizationcited by 315
  8. 2016Unsupervised Pixel-Level Domain Adaptation with Generative Adversarial Networkscited by 308
  9. 2017Size-Independent Sample Complexity of Neural Networkscited by 290
  10. 2017Positive-Unlabeled Learning with Non-Negative Risk Estimatorcited by 219
  11. 2018DeepJDOT: Deep Joint Distribution Optimal Transport for Unsupervised Domain Adaptationcited by 214
  12. 2019Adaptive Gradient-Based Meta-Learning Methodscited by 183
  13. 2019Deep ReLU Networks Have Surprisingly Few Activation Patternscited by 161
  14. 2019A review of domain adaptation without target labelscited by 156
  15. 2020Lower Bounds and Optimal Algorithms for Personalized Federated Learningcited by 123
  16. 2020Tilted Empirical Risk Minimizationcited by 106
  17. 2020Rethinking Importance Weighting for Deep Learning under Distribution Shiftcited by 93
  18. 2020Deep Visual Domain Adaptationcited by 91
  19. 2020Understanding Double Descent Requires a Fine-Grained Bias-Variance Decompositioncited by 76
  20. 2016Harder, Better, Faster, Stronger Convergence Rates for Least-Squares Regressioncited by 64
  21. 2021What Do We Mean by Generalization in Federated Learning?cited by 62
  22. 2020Non-Negative Bregman Divergence Minimization for Deep Direct Density Ratio Estimationcited by 45
  23. 2020Adaptive Gradient Quantization for Data-Parallel SGDcited by 45
  24. 2021The Benefits of Implicit Regularization from SGD in Least Squares Problemscited by 32
  25. 2019Mitigating Overfitting in Supervised Classification from Two Unlabeled Datasets: A Consistent Risk Correction Approachcited by 29
  26. 2020Off-Policy Evaluation and Learning for External Validity under a Covariate Shiftcited by 26
  27. 2020Information-Theoretic Bounds on the Generalization Error and Privacy Leakage in Federated Learningcited by 21
  28. 2020A One-step Approach to Covariate Shift Adaptationcited by 21
  29. 2022Mitigating Data Heterogeneity in Federated Learning with Data Augmentationcited by 21
  30. 2019NUQSGD: Provably Communication-efficient Data-parallel SGD via Nonuniform Quantizationcited by 18
  31. 2022FL Games: A Federated Learning Framework for Distribution Shiftscited by 14
  32. 2018Unsupervised Domain Adaptation using Regularized Hyper-graph Matchingcited by 13
  33. 2021FLIX: A Simple and Communication-Efficient Alternative to Local Methods in Federated Learningcited by 12
  34. 2020Importance Weight Estimation and Generalization in Domain Adaptation under Label Shiftcited by 10
  35. 2021Training on Test Data with Bayesian Adaptation for Covariate Shiftcited by 8

Built on by

Cited by year

Citing papers per year: 2024: 1; 2025: 5.

6 papers.

  1. 2025FedCF: Fair Federated Conformal Predictioncited by 3
  2. 2024Overcoming label shift with target-aware federated learningcited by 2
  3. 2025Addressing Label Shift in Distributed Learning via Entropy Regularizationcited by 1
  4. 2025Federated Learning via Meta-Variational Dropoutcited by 1
  5. 2025Shift Happens: Mixture of Experts based Continual Adaptation in Federated Learningcited by 0
  6. 2025Technical note on Fisher Information for Robust Federated Cross-Validationcited by 0