mlsanityData to 7 Oct 2026

The Skellam Mechanism for Differentially Private Federated Learning

arXiv 2110.04995 ↗First posted 11 Oct 2021cs.LG cs.CR cs.DScited by 66

Not on the map. Why?

Abstract (arXiv)

We introduce the multi-dimensional Skellam mechanism, a discrete differential privacy mechanism based on the difference of two independent Poisson random variables. To quantify its privacy guarantees, we analyze the privacy loss distribution via a numerical evaluation and provide a sharp bound on the R'enyi divergence between two shifted Skellam distributions. While useful in both centralized and distributed privacy applications, we investigate how it can be applied in the context of federated learning with secure aggregation under communication constraints. Our theoretical findings and extensive experimental evaluations demonstrate that the Skellam mechanism provides the same privacy-accuracy trade-offs as the continuous Gaussian mechanism, even when the precision is low. More importantly, Skellam is closed under summation and sampling from it only requires sampling from a Poisson distribution -- an efficient routine that ships with all machine learning and data analysis software packages. These features, along with its discrete nature and competitive privacy-accuracy trade-offs, make it an attractive practical alternative to the newly introduced discrete Gaussian mechanism.

Key facts. Cited by 66 papers in the map (data through 7 Oct 2026). Builds on 28 of its 50 references that are in the map. First posted on arXiv 11 Oct 2021.

Builds on

28 of 50 references.

References · 28

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  2. 2019Advances and Open Problems in Federated Learningcited by 3,021
  3. 2016Deep Learning with Differential Privacycited by 2,682
  4. 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,461
  5. 2020Adaptive Federated Optimizationcited by 1,036
  6. 2018LEAF: A Benchmark for Federated Settingscited by 780
  7. 2017Renyi Differential Privacycited by 643
  8. 2018Federated Learning for Mobile Keyboard Predictioncited by 565
  9. 2016Concentrated Differential Privacy: Simplifications, Extensions, and Lower Boundscited by 339
  10. 2021See through Gradients: Image Batch Recovery via GradInversioncited by 320
  11. 2013The Composition Theorem for Differential Privacycited by 290
  12. 2020Differentially Private Learning Needs Better Features (or Much More Data)cited by 233
16 more
  1. 2019Differentially Private Learning with Adaptive Clippingcited by 218
  2. 2018Improving the Gaussian Mechanism for Differential Privacy: Analytical Calibration and Optimal Denoisingcited by 216
  3. 2019R'enyi Differential Privacy of the Sampled Gaussian Mechanismcited by 199
  4. 2021Practical and Private (Deep) Learning without Sampling or Shufflingcited by 169
  5. 2018cpSGD: Communication-efficient and differentially-private distributed SGDcited by 157
  6. 2016Concentrated Differential Privacycited by 139
  7. 2020The Discrete Gaussian for Differential Privacycited by 134
  8. 2021The Distributed Discrete Gaussian Mechanism for Federated Learning with Secure Aggregationcited by 127
  9. 2019Federated Learning for Emoji Prediction in a Mobile Keyboardcited by 115
  10. 2014Differentially Private Empirical Risk Minimization: Efficient Algorithms and Tight Error Boundscited by 104
  11. 2018A General Approach to Adding Differential Privacy to Iterative Training Procedurescited by 84
  12. 2020Federated Learning via Posterior Averaging: A New Perspective and Practical Algorithmscited by 78
  13. 2020Training Production Language Models without Memorizing User Datacited by 68
  14. 2020A Better Bound Gives a Hundred Rounds: Enhanced Privacy Guarantees via f-Divergencescited by 21
  15. 2018Locally Private Bayesian Inference for Count Modelscited by 9
  16. 2017Computational Differential Privacy from Lattice-based Cryptographycited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 4; 2022: 17; 2023: 15; 2024: 19; 2025: 6; 2026: 5.

66 papers.

Sort:

  1. 2023How to DP-fy ML: A Practical Guide to Machine Learning with Differential Privacycited by 150
  2. 2021When the Curious Abandon Honesty: Federated Learning Is Not Privatecited by 126
  3. 2021Eluding Secure Aggregation in Federated Learning via Model Inconsistencycited by 66
  4. 2023Federated Learning of Gboard Language Models with Differential Privacycited by 64
  5. 2023Recent Advances on Federated Learning: A Systematic Surveycited by 40
  6. 2022The Fundamental Price of Secure Aggregation in Differentially Private Federated Learningcited by 39
  7. 2022On Privacy and Personalization in Cross-Silo Federated Learningcited by 36
  8. 2022Sparse Random Networks for Communication-Efficient Federated Learningcited by 32
  9. 2024PrE-Text: Training Language Models on Private Federated Data in the Age of LLMscited by 29
  10. 2023Privacy Amplification via Compression: Achieving the Optimal Privacy-Accuracy-Communication Trade-off in Distributed Mean Estimationcited by 22
  11. 2023Differentially Private Natural Language Models: Recent Advances and Future Directionscited by 20
  12. 2023Practical Differentially Private and Byzantine-resilient Federated Learningcited by 18
38 more
  1. 2022Privacy-Aware Compression for Federated Data Analysiscited by 17
  2. 2023z-SignFedAvg: A Unified Stochastic Sign-based Compression for Federated Learningcited by 17
  3. 2024The Privacy Power of Correlated Noise in Decentralized Learningcited by 17
  4. 2023Reconstructing Individual Data Points in Federated Learning Hardened with Differential Privacy and Secure Aggregationcited by 16
  5. 2023Randomized Quantization is All You Need for Differential Privacy in Federated Learningcited by 16
  6. 2024Federated Learning in Practice: Reflections and Projectionscited by 15
  7. 2022Distributed Differential Privacy in Multi-Armed Banditscited by 14
  8. 2023Is Aggregation the Only Choice? Federated Learning via Layer-wise Model Recombinationcited by 13
  9. 2021Towards Sparse Federated Analytics: Location Heatmaps under Distributed Differential Privacy with Secure Aggregationcited by 10
  10. 2022Federated Boosted Decision Trees with Differential Privacycited by 9
  11. 2023ULDP-FL: Federated Learning with Across Silo User-Level Differential Privacycited by 9
  12. 2022Pisces: Efficient Federated Learning via Guided Asynchronous Trainingcited by 8
  13. 2022Skellam Mixture Mechanism: a Novel Approach to Federated Learning with Differential Privacycited by 7
  14. 2022Privacy-Aware Compression for Federated Learning Through Numerical Mechanism Designcited by 6
  15. 2023Balancing Privacy Protection and Interpretability in Federated Learningcited by 6
  16. 2022Federated Calibration and Evaluation of Binary Classifierscited by 5
  17. 2023Private Federated Learning with Autotuned Compressioncited by 5
  18. 2024Confidential Federated Computationscited by 4
  19. 2024Improved Communication-Privacy Trade-offs in L2 Mean Estimation under Streaming Differential Privacycited by 4
  20. 2024Correlated Privacy Mechanisms for Differentially Private Distributed Mean Estimationcited by 4
  21. 2022FedEntropy: Efficient Device Grouping for Federated Learning Using Maximum Entropy Judgmentcited by 3
  22. 2022Brownian Noise Reduction: Maximizing Privacy Subject to Accuracy Constraintscited by 3
  23. 2023Breaking the Communication-Privacy-Accuracy Tradeoff with f-Differential Privacycited by 3
  24. 2024Uncertainty-Based Extensible Codebook for Discrete Federated Learning in Heterogeneous Data Siloscited by 3
  25. 2024Efficient and Scalable Implementation of Differentially Private Deep Learning without Shortcutscited by 3
  26. 2024DMM: Distributed Matrix Mechanism for Differentially-Private Federated Learning Based on Constant-Overhead Linear Secret Resharingcited by 3
  27. 2022Differentially private partitioned variational inferencecited by 2
  28. 2024TernaryVote: Differentially Private, Communication Efficient, and Byzantine Resilient Distributed Optimization on Heterogeneous Datacited by 2
  29. 2024Privacy Amplification for the Gaussian Mechanism via Bounded Supportcited by 2
  30. 2024FedSheafHN: Personalized Federated Learning on Graph-structured Datacited by 2
  31. 2025Communication-Efficient and Privacy-Adaptable Mechanism for Federated Learningcited by 2
  32. 2023Cross-Silo Federated Learning Across Divergent Domains with Iterative Parameter Alignmentcited by 1
  33. 2024Decomposable Submodular Maximization in Federated Settingcited by 1
  34. 2024Privacy-Aware Randomized Quantization via Linear Programmingcited by 1
  35. 2024Universally Harmonizing Differential Privacy Mechanisms for Federated Learning: Boosting Accuracy and Convergencecited by 1
  36. 2024On Using Secure Aggregation in Differentially Private Federated Learning with Multiple Local Stepscited by 1
  37. 2025Personalized Subgraph Federated Learning with Sheaf Collaborationcited by 1
  38. 2026Communication-Efficient and Privacy-Adaptable Mechanism -- a Federated Learning Scheme with Convergence Analysiscited by 1

50 most cited of 66 All 66 (JSON)