mlsanityData to 14 Oct 2026

Private Aggregation for Byzantine-Resilient Heterogeneous Federated Learning

arXiv 2506.09870 ↗First posted 11 Jun 2025cs.LG cs.DC cs.ITcited by 0

Not on the map. Why?

Abstract (arXiv)

Ensuring resilience to Byzantine clients while maintaining the privacy of the clients' data is a fundamental challenge in federated learning (FL). When the clients' data is homogeneous, suitable countermeasures were studied from an information-theoretic perspective utilizing secure aggregation techniques while ensuring robust aggregation of the clients' gradients. However, the countermeasures used fail when the clients' data is heterogeneous. Suitable pre-processing techniques, such as nearest neighbor mixing, were recently shown to enhance the performance of those countermeasures in the heterogeneous setting. Nevertheless, those pre-processing techniques cannot be applied with the introduced privacy-preserving mechanisms. We propose a multi-stage method encompassing a careful co-design of verifiable secret sharing, secure aggregation, and a tailored symmetric private information retrieval scheme to achieve information-theoretic privacy guarantees and Byzantine resilience under data heterogeneity. We evaluate the effectiveness of our scheme on a variety of attacks and show how it outperforms the previously known techniques. Since the communication overhead of secure aggregation is non-negligible, we investigate the interplay with zero-order estimation methods that reduce the communication cost in state-of-the-art FL tasks and thereby make private aggregation scalable.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 33 of its 48 references that are in the map. First posted on arXiv 11 Jun 2025.

Builds on

33 of 48 references.

References · 33

  1. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  3. 2018Byzantine-Robust Distributed Learning: Towards Optimal Statistical Ratescited by 711
  4. 2015Deep Learning with Limited Numerical Precisioncited by 376
  5. 2020FLTrust: Byzantine-robust Federated Learning via Trust Bootstrappingcited by 346
  6. 2023Fine-Tuning Language Models with Just Forward Passescited by 345
  7. 2019Robust Aggregation for Federated Learningcited by 317
  8. 2019A Little Is Enough: Circumventing Defenses For Distributed Learningcited by 288
  9. 2019Fall of Empires: Breaking Byzantine-tolerant SGD by Inner Product Manipulationcited by 141
  10. 2020Learning from History for Byzantine Robust Optimizationcited by 137
  11. 2020Byzantine-Robust Learning on Heterogeneous Datasets via Bucketingcited by 128
  12. 2020Byzantine-Resilient Secure Federated Learningcited by 93
21 more
  1. 2016Learning from Untrusted Datacited by 90
  2. 2022Survey on Federated Learning Threats: concepts, taxonomy on attacks and defences, experimental study and challengescited by 84
  3. 2020Defending against Backdoors in Federated Learning with Robust Learning Ratecited by 82
  4. 2020Turbo-Aggregate: Breaking the Quadratic Aggregation Barrier in Secure Federated Learningcited by 78
  5. 2023Fixing by Mixing: A Recipe for Optimal Byzantine ML under Heterogeneitycited by 65
  6. 2021LightSecAgg: a Lightweight and Versatile Design for Secure Aggregation in Federated Learningcited by 64
  7. 2017Distributed Statistical Machine Learning in Adversarial Settings: Byzantine Gradient Descentcited by 59
  8. 2020Byzantine-Resilient Non-Convex Stochastic Gradient Descentcited by 57
  9. 2020Collaborative Learning in the Jungle (Decentralized, Byzantine, Heterogeneous, Asynchronous and Nonconvex Learning)cited by 54
  10. 2021Approximate Byzantine Fault-Tolerance in Distributed Optimizationcited by 28
  11. 2021CodedPaddedFL and CodedSecAgg: Straggler Mitigation and Secure Aggregation in Federated Learningcited by 20
  12. 2022SwiftAgg+: Achieving Asymptotically Optimal Communication Loads in Secure Aggregation for Federated Learningcited by 18
  13. 2022Byzantines can also Learn from History: Fall of Centered Clipping in Federated Learningcited by 8
  14. 2024Byzantine-Resilient Secure Aggregation for Federated Learning Without Privacy Compromisescited by 7
  15. 2024Secure Aggregation is Not Private Against Membership Inference Attackscited by 6
  16. 2023Private Aggregation in Hierarchical Wireless Federated Learning with Partial and Full Collusioncited by 5
  17. 2023ByzSecAgg: A Byzantine-Resistant Secure Aggregation Scheme for Federated Learning Based on Coded Computing and Vector Commitmentcited by 4
  18. 2024Communication-Efficient Byzantine-Resilient Federated Zero-Order Optimizationcited by 4
  19. 2025Byzantine-Resilient Zero-Order Optimization for Communication-Efficient Heterogeneous Federated Learningcited by 4
  20. 2024LoByITFL: Low Communication Secure and Private Federated Learningcited by 3
  21. 2024PriRoAgg: Achieving Robust Model Aggregation with Minimum Privacy Leakage for Federated Learningcited by 3

Built on by

Not cited yet.