mlsanityData to 14 Oct 2026

Private Federated Learning with Autotuned Compression

arXiv 2307.10999 ↗First posted 20 Jul 2023cs.LG stat.MLcited by 5

Not on the map. Why?

Abstract (arXiv)

We propose new techniques for reducing communication in private federated learning without the need for setting or tuning compression rates. Our on-the-fly methods automatically adjust the compression rate based on the error induced during training, while maintaining provable privacy guarantees through the use of secure aggregation and differential privacy. Our techniques are provably instance-optimal for mean estimation, meaning that they can adapt to the ``hardness of the problem" with minimal interactivity. We demonstrate the effectiveness of our approach on real-world datasets by achieving favorable compression rates without the need for tuning.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 47 of its 69 references that are in the map. First posted on arXiv 20 Jul 2023.

Builds on

47 of 69 references.

References · 47

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  2. 2016Deep Learning with Differential Privacycited by 2,728
  3. 2017Mixed Precision Trainingcited by 1,156
  4. 2017Renyi Differential Privacycited by 650
  5. 2018Sparsified SGD with Memorycited by 376
  6. 2017TernGrad: Ternary Gradients to Reduce Communication in Distributed Deep Learningcited by 256
  7. 2017Sparse Communication for Distributed Gradient Descentcited by 249
  8. 2019Private Stochastic Convex Optimization with Optimal Ratescited by 186
  9. 2020FetchSGD: Communication-Efficient Federated Learning with Sketchingcited by 179
  10. 2016Practical Secure Aggregation for Federated Learning on User-Held Datacited by 178
  11. 2021Practical and Private (Deep) Learning without Sampling or Shufflingcited by 170
  12. 2018cpSGD: Communication-efficient and differentially-private distributed SGDcited by 158
35 more
  1. 2021The Distributed Discrete Gaussian Mechanism for Federated Learning with Secure Aggregationcited by 128
  2. 2018Sever: A Robust Meta-Algorithm for Stochastic Optimizationcited by 125
  3. 2021Robbing the Fed: Directly Obtaining Private Data in Federated Learning with Modified Modelscited by 108
  4. 2016Minimax Optimal Procedures for Locally Private Estimationcited by 107
  5. 2019A Short Note on Concentration Inequalities for Random Vectors with SubGaussian Normcited by 106
  6. 2017Finite Sample Differentially Private Confidence Intervalscited by 104
  7. 2018LAG: Lazily Aggregated Gradient for Communication-Efficient Distributed Learningcited by 96
  8. 2019Communication-efficient distributed SGD with Sketchingcited by 81
  9. 2018Robust Estimation via Robust Gradient Estimationcited by 76
  10. 2020CoinPress: Practical Private Mean and Covariance Estimationcited by 74
  11. 2020Private Mean Estimation of Heavy-Tailed Distributionscited by 73
  12. 2020Breaking the Communication-Privacy-Accuracy Trilemmacited by 68
  13. 2021The Skellam Mechanism for Differentially Private Federated Learningcited by 68
  14. 2021Robust and Differentially Private Mean Estimationcited by 66
  15. 2023Federated Learning of Gboard Language Models with Differential Privacycited by 66
  16. 2021Differential privacy and robust statistics in high dimensionscited by 64
  17. 2021Adaptive Quantization of Model Updates for Communication-Efficient Federated Learningcited by 63
  18. 2015Between Pure and Approximate Differential Privacycited by 56
  19. 2019Understanding Top-k Sparsification in Distributed Deep Learningcited by 56
  20. 2019Average-Case Averages: Private Algorithms for Smooth Sensitivity and Mean Estimationcited by 52
  21. 2022Multi-Epoch Matrix Factorization Mechanisms for Private Machine Learningcited by 48
  22. 2022Do Gradient Inversion Attacks Make Federated Learning Unsafe?cited by 44
  23. 2022The Fundamental Price of Secure Aggregation in Differentially Private Federated Learningcited by 39
  24. 2022Optimal Algorithms for Mean Estimation under Local Differential Privacycited by 36
  25. 2022Faster Rates of Convergence to Stationary Points in Differentially Private Optimizationcited by 32
  26. 2020A Primer on Private Statisticscited by 30
  27. 2021Fast and Memory Efficient Differentially Private-SGD via JL Projectionscited by 29
  28. 2021Lossless Compression of Efficient Private Local Randomizerscited by 27
  29. 2020FedSKETCH: Communication-Efficient and Private Federated Learning via Sketchingcited by 20
  30. 2022Differentially Private Generalized Linear Models Revisitedcited by 17
  31. 2022Adaptive Compression for Communication-Efficient Distributed Trainingcited by 16
  32. 2022Correlated quantization for distributed mean estimation and optimizationcited by 14
  33. 2020Locally private non-asymptotic testing of discrete distributions is faster using interactive mechanismscited by 11
  34. 2022Improved Utility Analysis of Private CountSketchcited by 6
  35. 2022Two Models are Better than One: Federated Learning Is Not Private For Google GBoard Next Word Predictioncited by 4

Built on by

Cited by year

Citing papers per year: 2023: 2; 2024: 2; 2025: 1.

5 papers.

  1. 2024Secure Aggregation is Not Private Against Membership Inference Attackscited by 6
  2. 2024FedPeWS: Personalized Warmup via Subnetworks for Enhanced Heterogeneous Federated Learningcited by 2
  3. 2025Differential Privacy Analysis of Decentralized Gossip Averaging under Varying Threat Modelscited by 2
  4. 2023ALI-DPFL: Differentially Private Federated Learning with Adaptive Local Iterationscited by 1
  5. 2023Robust and Actively Secure Serverless Collaborative Learningcited by 1