mlsanityData to 7 Oct 2026

Private Federated Submodel Learning with Sparsification

arXiv 2205.15992 ↗First posted 31 May 2022cs.IT cs.CR cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

We investigate the problem of private read update write (PRUW) in federated submodel learning (FSL) with sparsification. In FSL, a machine learning model is divided into multiple submodels, where each user updates only the submodel that is relevant to the user's local data. PRUW is the process of privately performing FSL by reading from and writing to the required submodel without revealing the submodel index, or the values of updates to the databases. Sparsification is a widely used concept in learning, where the users update only a small fraction of parameters to reduce the communication cost. Revealing the coordinates of these selected (sparse) updates leaks privacy of the user. We show how PRUW in FSL can be performed with sparsification. We propose a novel scheme which privately reads from and writes to arbitrary parameters of any given submodel, without revealing the submodel index, values of updates, or the coordinates of the sparse updates, to databases. The proposed scheme achieves significantly lower reading and writing costs compared to what is achieved without sparsification.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 39 of its 56 references that are in the map. First posted on arXiv 31 May 2022.

Builds on

39 of 56 references.

References · 39

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  2. 2019Advances and Open Problems in Federated Learningcited by 3,021
  3. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,221
  4. 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,531
  5. 2019Deep Leakage from Gradientscited by 1,004
  6. 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 847
  7. 2020Inverting Gradients -- How easy is it to break privacy in federated learning?cited by 707
  8. 2018Exploiting Unintended Feature Leakage in Collaborative Learningcited by 509
  9. 2017Differentially Private Federated Learning: A Client Level Perspectivecited by 451
  10. 2019Federated Machine Learning: Concept and Applicationscited by 409
  11. 2019FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantizationcited by 349
  12. 2018Beyond Inferring Class Representatives: User-Level Privacy Leakage From Federated Learningcited by 242
27 more
  1. 2018Privacy Amplification by Subsampling: Tight Analyses via Couplings and Divergencescited by 237
  2. 2018The Convergence of Sparsified Gradient Methodscited by 228
  3. 2017Gradient Sparsification for Communication-Efficient Distributed Optimizationcited by 199
  4. 2019Qsparse-local-SGD: Distributed SGD with Quantization, Sparsification, and Local Computationscited by 179
  5. 2018Amplification by Shuffling: From Local to Central Differential Privacy via Anonymitycited by 163
  6. 2018cpSGD: Communication-efficient and differentially-private distributed SGDcited by 157
  7. 2016QSGD: Communication-Efficient SGD via Gradient Quantization and Encodingcited by 147
  8. 2020Adaptive Gradient Sparsification for Efficient Federated Learning: An Online Learning Approachcited by 80
  9. 2019The Privacy Blanket of the Shuffle Modelcited by 73
  10. 2020rTop-k: A Statistical Estimation Approach to Distributed SGDcited by 27
  11. 2021Time-Correlated Sparsification for Communication-Efficient Federated Learningcited by 25
  12. 2021Private Retrieval, Computing and Learning: Recent Progress and Future Challengescited by 12
  13. 2020Locally Private Distributed Reinforcement Learningcited by 9
  14. 2022Time-Correlated Sparsification for Efficient Over-the-Air Model Aggregation in Wireless Federated Learningcited by 7
  15. 2016The Capacity of Private Information Retrievalcited by 6
  16. 2016The Capacity of Robust Private Information Retrieval with Colluding Databasescited by 6
  17. 2016The Capacity of Private Information Retrieval from Coded Databasescited by 6
  18. 2016The Capacity of Symmetric Private Information Retrievalcited by 5
  19. 2017Multi-Message Private Information Retrieval: Capacity Results and Near-Optimal Schemescited by 5
  20. 2018Capacity-Achieving Private Information Retrieval Codes with Optimal Message Size and Upload Costcited by 4
  21. 2020Semantic Private Information Retrievalcited by 4
  22. 2017The Capacity of Private Information Retrieval from Byzantine and Colluding Databasescited by 3
  23. 2017Private Information Retrieval with Side Informationcited by 3
  24. 2019Secure Federated Submodel Learningcited by 2
  25. 2019Private Set Intersection: A Multi-Message Symmetric Private Information Retrieval Perspectivecited by 1
  26. 2020Asymmetric Leaky Private Information Retrievalcited by 1
  27. 2022Private Read Update Write (PRUW) with Storage Constrained Databasescited by 1

Built on by

Not cited yet.