mlsanityData to 7 Oct 2026

Private Regression via Data-Dependent Sufficient Statistic Perturbation

arXiv 2405.15002 ↗First posted 23 May 2024cs.LG stat.MLcited by 4

Not on the map. Why?

Abstract (arXiv)

Sufficient statistic perturbation (SSP) is a widely used method for differentially private linear regression. SSP adopts a data-independent approach where privacy noise from a simple distribution is added to sufficient statistics. However, sufficient statistics can often be expressed as linear queries and better approximated by data-dependent mechanisms. In this paper we introduce data-dependent SSP for linear regression based on post-processing privately released marginals, and find that it outperforms state-of-the-art data-independent SSP. We extend this result to logistic regression by developing an approximate objective that can be expressed in terms of sufficient statistics, resulting in a novel and highly competitive SSP approach for logistic regression. We also make a connection to synthetic data for machine learning: for models with sufficient statistics, training on synthetic data corresponds to data-dependent SSP, with the overall utility determined by how well the mechanism answers these linear queries.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 30 of its 59 references that are in the map. First posted on arXiv 23 May 2024.

Builds on

30 of 59 references.

References · 30

  1. 2016Deep Learning with Differential Privacycited by 2,682
  2. 2021Retiring Adult: New Datasets for Fair Machine Learningcited by 443
  3. 2016Concentrated Differential Privacy: Simplifications, Extensions, and Lower Boundscited by 339
  4. 2018Differentially Private Generative Adversarial Networkcited by 235
  5. 2023How to DP-fy ML: A Practical Guide to Machine Learning with Differential Privacycited by 150
  6. 2021Hyperparameter Tuning with Renyi Differential Privacycited by 104
  7. 2021Winning the NIST Contest: A scalable and general approach to differentially private synthetic datacited by 101
  8. 2022AIM: An Adaptive and Iterative Mechanism for Differentially Private Synthetic Datacited by 88
  9. 2021Benchmarking Differentially Private Synthetic Data Generation Algorithmscited by 80
  10. 2019Graphical-model based estimation and inference for differential privacycited by 73
  11. 2015Privacy for Free: Posterior Sampling and Stochastic Gradient Monte Carlocited by 61
  12. 2018Revisiting differentially private linear regression: optimal and adaptive prediction & estimation in unbounded domaincited by 60
18 more
  1. 2021Iterative Methods for Private Synthetic Data: Unifying Framework and New Methodscited by 58
  2. 2021Differentially Private Query Release Through Adaptive Projectioncited by 50
  3. 2020New Oracle-Efficient Algorithms for Private Synthetic Data Releasecited by 43
  4. 2022Composition of Differential Privacy & Privacy Amplification by Subsamplingcited by 42
  5. 2015Differentially Private Ordinary Least Squarescited by 36
  6. 2020Differentially Private Synthetic Data: Applied Evaluations and Enhancementscited by 36
  7. 2022Private Synthetic Data for Multitask Learning and Marginal Queriescited by 35
  8. 2016On the Theory and Practice of Privacy-Preserving Bayesian Data Analysiscited by 34
  9. 2014Dual Query: Practical Private Query Release for High Dimensional Datacited by 26
  10. 2019Differentially Private Bayesian Linear Regressioncited by 24
  11. 2020Parametric Bootstrap for Differentially Private Confidence Intervalscited by 19
  12. 2022Easy Differentially Private Linear Regressioncited by 19
  13. 2020Differentially Private Bayesian Inference for Generalized Linear Modelscited by 16
  14. 2017PASS-GLM: polynomial approximate sufficient statistics for scalable Bayesian GLM inferencecited by 11
  15. 2021HDMM: Optimizing error of high-dimensional statistical queries under differential privacycited by 11
  16. 2024Private Gradient Descent for Linear Regression: Tighter Error Bounds and Instance-Specific Uncertainty Estimationcited by 11
  17. 2023Improved Differentially Private Regression via Gradient Boostingcited by 6
  18. 2024Bounding the Excess Risk for Linear Models Trained on Marginal-Preserving, Differentially-Private, Synthetic Datacited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 2.

4 papers.

  1. 2025The Gaussian Mixing Mechanism: Renyi Differential Privacy via Gaussian Sketchescited by 2
  2. 2026Near-Optimal Private Linear Regression via Iterative Hessian Mixingcited by 1
  3. 2025Differentially Private Two-Stage Gradient Descent for Instrumental Variable Regressioncited by 0
  4. 2026Refined Differentially Private Linear Regression via Extension of a Free Lunch Resultcited by 0