mlsanityData to 14 Oct 2026

Certifiably Robust Model Evaluation in Federated Learning under Meta-Distributional Shifts

arXiv 2410.20250 ↗First posted 26 Oct 2024stat.ML cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

We address the challenge of certifying the performance of a federated learning model on an unseen target network using only measurements from the source network that trained the model. Specifically, consider a source network "A" with K clients, each holding private, non-IID datasets drawn from heterogeneous distributions, modeled as samples from a broader meta-distribution μ. Our goal is to provide certified guarantees for the model's performance on a different, unseen network "B", governed by an unknown meta-distribution μ', assuming the deviation between μ and μ' is bounded either in Wasserstein distance or an f-divergence. We derive worst-case uniform guarantees for both the model's average loss and its risk CDF, the latter corresponding to a novel, adversarially robust version of the Dvoretzky-Kiefer-Wolfowitz (DKW) inequality. In addition, we show how the vanilla DKW bound enables principled certification of the model's true performance on unseen clients within the same (source) network. Our bounds are efficiently computable, asymptotically minimax optimal, and preserve clients' privacy. We also establish non-asymptotic generalization bounds that converge to zero as K grows and the minimum per-client sample size exceeds O(log K). Empirical evaluations confirm the practical utility of our bounds across real-world tasks. The project code is available at: github.com/samin-mehdizadeh/Robust-Evaluation-DKW

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 31 of its 52 references that are in the map. First posted on arXiv 26 Oct 2024.

Builds on

31 of 52 references.

References · 31

  1. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  3. 2021Convex optimizationcited by 2,515
  4. 2013Stochastic First- and Zeroth-order Methods for Nonconvex Stochastic Programmingcited by 724
  5. 2020Federated Learning with Matched Averagingcited by 564
  6. 2021FedProto: Federated Prototype Learning across Heterogeneous Clientscited by 352
  7. 2017Certifying Some Distributional Robustness with Principled Adversarial Trainingcited by 337
  8. 2021No Fear of Heterogeneity: Classifier Calibration for Federated Learning with Non-IID Datacited by 241
  9. 2023Heterogeneous Federated Learning: State-of-the-art and Research Challengescited by 161
  10. 2020Robust Federated Learning: The Case of Affine Distribution Shiftscited by 102
  11. 2021What Do We Mean by Generalization in Federated Learning?cited by 62
  12. 2020Maximum Mean Discrepancy Test is Aware of Adversarial Attackscited by 44
19 more
  1. 2023Recent Advances on Federated Learning: A Systematic Surveycited by 41
  2. 2022MetaFed: Federated Learning among Federations with Cyclic Knowledge Distillation for Personalized Healthcarecited by 37
  3. 2020Device Heterogeneity in Federated Learning: A Superquantile Approachcited by 21
  4. 2021Curse or Redemption? How Data Heterogeneity Affects the Robustness of Federated Learningcited by 21
  5. 2023Delving into the Adversarial Robustness of Federated Learningcited by 19
  6. 2020Adversarial Robustness through Bias Variance Decomposition: A New Perspective for Federated Learningcited by 13
  7. 2022A Short and General Duality Proof for Wasserstein Distributionally Robust Optimizationcited by 12
  8. 2022An Optimal Transport Approach to Personalized Federated Learningcited by 12
  9. 2022Distributionally Adaptive Meta Reinforcement Learningcited by 11
  10. 2022Superquantiles at Work: Machine Learning Applications and Efficient Subgradient Computationcited by 10
  11. 2021Generalization Bounds with Minimal Dependency on Hypothesis Class via Distributionally Robust Optimizationcited by 9
  12. 2024FedLPS: Heterogeneous Federated Learning for Multiple Tasks with Local Parameter Sharingcited by 8
  13. 2021A Wasserstein Minimax Framework for Mixed Linear Regressioncited by 7
  14. 2023A Survey of Federated Evaluation in Federated Learningcited by 7
  15. 2024Beyond the Federation: Topology-aware Federated Learning for Generalization to Unseen Clientscited by 7
  16. 2023Distributionally Robust Optimization with Bias and Variance Reductioncited by 6
  17. 2022Federated Learning Aggregation: New Robust Algorithms with Guaranteescited by 4
  18. 2022Sample Complexity Bounds for Learning High-dimensional Simplices in Noisy Regimescited by 4
  19. 2023Federated Experiment Design under Distributed Differential Privacycited by 3

Built on by

Not cited yet.