mlsanityData to 14 Oct 2026

Byzantine-Resilient Federated PCA and Low Rank Column-wise Sensing

arXiv 2309.14512 ↗First posted 25 Sep 2023cs.IT math.IT stat.MLcited by 0

Not on the map. Why?

Abstract (arXiv)

This work considers two related learning problems in a federated attack prone setting: federated principal components analysis (PCA) and federated low rank column-wise sensing (LRCS). The node attacks are assumed to be Byzantine which means that the attackers are omniscient and can collude. We introduce a novel provably Byzantine-resilient communication-efficient and sampleefficient algorithm, called Subspace-Median, that solves the PCA problem and is a key part of the solution for the LRCS problem. We also study the most natural Byzantine-resilient solution for federated PCA, a geometric median based modification of the federated power method, and explain why it is not useful. Our second main contribution is a complete alternating gradient descent (GD) and minimization (altGDmin) algorithm for Byzantine-resilient horizontally federated LRCS and sample and communication complexity guarantees for it. Extensive simulation experiments are used to corroborate our theoretical guarantees. The ideas that we develop for LRCS are easily extendable to other LR recovery problems as well.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 32 of its 68 references that are in the map. First posted on arXiv 25 Sep 2023.

Builds on

32 of 68 references.

References · 32

  1. 2019Advances and Open Problems in Federated Learningcited by 3,046
  2. 2018Byzantine-Robust Distributed Learning: Towards Optimal Statistical Ratescited by 711
  3. 2021Exploiting Shared Representations for Personalized Federated Learningcited by 541
  4. 2014SAGA: A Fast Incremental Gradient Method With Support for Non-Strongly Convex Composite Objectivescited by 402
  5. 2020FLTrust: Byzantine-robust Federated Learning via Trust Bootstrappingcited by 346
  6. 2018The Hidden Vulnerability of Distributed Learning in Byzantiumcited by 329
  7. 2019Robust Aggregation for Federated Learningcited by 317
  8. 2020Few-Shot Learning via Learning the Representation, Provablycited by 214
  9. 2016Robust Estimators in High Dimensions without the Computational Intractabilitycited by 209
  10. 2018RSA: Byzantine-Robust Stochastic Aggregation Methods for Distributed Learning from Heterogeneous Datasetscited by 149
  11. 2016Agnostic Estimation of Mean and Covariancecited by 131
  12. 2020Spectral Methods for Data Science: A Statistical Perspectivecited by 112
20 more
  1. 2019Recent Advances in Algorithmic High-Dimensional Robust Statisticscited by 101
  2. 2019Robust Federated Learning in a Heterogeneous Environmentcited by 90
  3. 2017Being Robust (in High Dimensions) Can Be Practicalcited by 83
  4. 2018Byzantine Stochastic Gradient Descentcited by 81
  5. 2019Federated Variance-Reduced Stochastic Gradient Descent with Robustness to Byzantine Attackscited by 79
  6. 2018DRACO: Byzantine-resilient Distributed Training via Redundant Gradientscited by 75
  7. 2013The Noisy Power Method: A Meta Algorithm with Applicationscited by 58
  8. 2020Byzantine-Resilient Non-Convex Stochastic Gradient Descentcited by 57
  9. 2018Federated Learning in Distributed Medical Databases: Meta-Analysis of Large-Scale Subcortical Brain Datacited by 46
  10. 2019DETOX: A Redundancy-based Framework for Faster and More Robust Gradient Aggregationcited by 44
  11. 2016Fast Algorithms for Robust PCA via Gradient Descentcited by 39
  12. 2016Geometric Median in Nearly Linear Timecited by 39
  13. 2017Robust Subspace Learning: Robust PCA, Robust Subspace Tracking, and Robust Subspace Recoverycited by 25
  14. 2021Robust Training in High Dimensions via Block Coordinate Geometric Median Descentcited by 20
  15. 2020Byzantine-Resilient SGD in High Dimensions on Heterogeneous Datacited by 17
  16. 2021Turning Channel Noise into an Accelerator for Over-the-Air Principal Component Analysiscited by 14
  17. 2017Nonconvex Low-Rank Matrix Recovery with Arbitrary Outliers via Median-Truncated Gradient Descentcited by 10
  18. 2017Provable Dynamic Robust PCA or Robust Subspace Trackingcited by 5
  19. 2020Fast Robust Subspace Tracking via PCA in Sparse Data-Dependent Noisecited by 3
  20. 2020Federated Over-Air Subspace Tracking from Incomplete and Corrupted Datacited by 2

Built on by

Not cited yet.