mlsanityData to 14 Oct 2026

Approximate Gradient Coding for Privacy-Flexible Federated Learning with Non-IID Data

arXiv 2404.03524 ↗First posted 4 Apr 2024cs.LG cs.CR cs.DCcited by 0

Not on the map. Why?

Abstract (arXiv)

This work focuses on the challenges of non-IID data and stragglers/dropouts in federated learning. We introduce and explore a privacy-flexible paradigm that models parts of the clients' local data as non-private, offering a more versatile and business-oriented perspective on privacy. Within this framework, we propose a data-driven strategy for mitigating the effects of label heterogeneity and client straggling on federated learning. Our solution combines both offline data sharing and approximate gradient coding techniques. Through numerical simulations using the MNIST dataset, we demonstrate that our approach enables achieving a deliberate trade-off between privacy and utility, leading to improved model convergence and accuracy while using an adaptable portion of non-private data.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 37 of its 47 references that are in the map. First posted on arXiv 4 Apr 2024.

Builds on

37 of 47 references.

References · 37

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  2. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,183
  3. 2019Advances and Open Problems in Federated Learningcited by 3,046
  4. 2018Federated Learning with Non-IID Datacited by 1,108
  5. 2020Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimizationcited by 906
  6. 2019Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classificationcited by 851
  7. 2021Model-Contrastive Federated Learningcited by 766
  8. 2017Federated Multi-Task Learningcited by 672
  9. 2020Ditto: Fair and Robust Federated Learning Through Personalizationcited by 672
  10. 2018Local SGD Converges Fast and Communicates Littlecited by 605
  11. 2020Federated Learning with Matched Averagingcited by 564
  12. 2021Federated Learning Based on Dynamic Regularizationcited by 562
25 more
  1. 2019Bayesian Nonparametric Federated Learning of Neural Networkscited by 387
  2. 2020Adaptive Personalized Federated Learningcited by 358
  3. 2020Think Locally, Act Globally: Federated Learning with Local and Global Representationscited by 325
  4. 2020Three Approaches for Personalization with Applications to Federated Learningcited by 325
  5. 2018Communication-Efficient On-Device Machine Learning: Federated Distillation and Augmentation under Non-IID Private Datacited by 243
  6. 2021Linear Convergence in Federated Learning: Tackling Client Heterogeneity and Sparse Gradientscited by 123
  7. 2021FedMix: Approximation of Mixup under Mean Augmented Federated Learningcited by 108
  8. 2017On the convergence properties of a K-step averaging stochastic gradient descent algorithm for nonconvex optimizationcited by 92
  9. 2020FedGAN: Federated Generative Adversarial Networks for Distributed Datacited by 90
  10. 2015Speeding Up Distributed Machine Learning Using Codescited by 76
  11. 2021Towards Fair Federated Learning with Zero-Shot Data Augmentationcited by 64
  12. 2020Federated Learning via Synthetic Datacited by 54
  13. 2022Towards Understanding and Mitigating Dimensional Collapse in Heterogeneous Federated Learningcited by 46
  14. 2019Hybrid-FL for Wireless Networks: Cooperative Learning Mechanism Using Non-IID Datacited by 38
  15. 2020On the Outsized Importance of Learning Rates in Local Update Methodscited by 36
  16. 2018Graph Oracle Models, Lower Bounds, and Gaps for Parallel Stochastic Optimizationcited by 34
  17. 2020Coded Computing for Low-Latency Federated Learning over Wireless Edge Networkscited by 32
  18. 2018Communication-Computation Efficient Gradient Codingcited by 27
  19. 2017Straggler Mitigation in Distributed Optimization Through Data Encodingcited by 26
  20. 2019ErasureHead: Distributed Gradient Descent without Delays Using Approximate Gradient Codingcited by 22
  21. 2021CodedPaddedFL and CodedSecAgg: Straggler Mitigation and Secure Aggregation in Federated Learningcited by 20
  22. 2019Fundamental Limits of Approximate Gradient Codingcited by 15
  23. 2020Straggler-aware Distributed Learning: Communication Computation Latency Trade-offcited by 15
  24. 2020Approximate Gradient Coding with Optimal Decodingcited by 14
  25. 2017Approximate Gradient Coding via Sparse Random Graphscited by 12

Built on by

Not cited yet.