mlsanityData to 7 Oct 2026

Differentially private inference via noisy optimization

arXiv 2103.11003 ↗First posted 19 Mar 2021math.ST cs.CR cs.LGcited by 27

Not on the map. Why?

Abstract (arXiv)

We propose a general optimization-based framework for computing differentially private M-estimators and a new method for constructing differentially private confidence regions. Firstly, we show that robust statistics can be used in conjunction with noisy gradient descent or noisy Newton methods in order to obtain optimal private estimators with global linear or quadratic convergence, respectively. We establish local and global convergence guarantees, under both local strong convexity and self-concordance, showing that our private estimators converge with high probability to a small neighborhood of the non-private M-estimators. Secondly, we tackle the problem of parametric inference by constructing differentially private estimators of the asymptotic variance of our private M-estimators. This naturally leads to approximate pivotal statistics for constructing confidence regions and conducting hypothesis testing. We demonstrate the effectiveness of a bias correction that leads to enhanced small-sample empirical performance in simulations. We illustrate the benefits of our methods in several numerical examples.

Key facts. Cited by 27 papers in the map (data through 7 Oct 2026). Builds on 32 of its 81 references that are in the map. First posted on arXiv 19 Mar 2021.

Builds on

32 of 81 references.

References · 32

  1. 2016Deep Learning with Differential Privacycited by 2,682
  2. 2014Convex Optimization: Algorithms and Complexitycited by 569
  3. 2015An Introduction to Matrix Concentration Inequalitiescited by 391
  4. 2014RAPPOR: Randomized Aggregatable Privacy-Preserving Ordinal Responsecited by 262
  5. 2019Private Stochastic Convex Optimization with Optimal Ratescited by 183
  6. 2020Private Stochastic Convex Optimization: Optimal Rates in Linear Timecited by 152
  7. 2017Collecting Telemetry Data Privatelycited by 147
  8. 2018Differentially Private Empirical Risk Minimization Revisited: Faster and More Generalcited by 127
  9. 2019Deep Learning with Gaussian Differential Privacycited by 112
  10. 2016Minimax Optimal Procedures for Locally Private Estimationcited by 103
  11. 2017Finite Sample Differentially Private Confidence Intervalscited by 103
  12. 2013Regularized M-estimators with nonconvexity: Statistical and algorithmic theory for local optimacited by 66
20 more
  1. 2015Privacy for Free: Posterior Sampling and Stochastic Gradient Monte Carlocited by 61
  2. 2018Revisiting differentially private linear regression: optimal and adaptive prediction & estimation in unbounded domaincited by 60
  3. 2017Privacy Loss in Apple's Implementation of Differential Privacy on MacOS 10.12cited by 42
  4. 2020Privacy Amplification via Random Check-Inscited by 39
  5. 2016Stochastic Quasi-Newton Methods for Nonconvex Stochastic Optimizationcited by 37
  6. 2015Differentially Private Ordinary Least Squarescited by 36
  7. 2017Generalized Self-Concordant Functions: A Recipe for Newton-Type Methodscited by 33
  8. 2018Concentrated Differentially Private Gradient Descent with Adaptive per-Iteration Privacy Budgetcited by 31
  9. 2018Finite-sample analysis of M-estimators using self-concordancecited by 30
  10. 2015Statistical consistency and asymptotic normality for high-dimensional robust M-estimatorscited by 27
  11. 2017Differentially Private Testing of Identity and Closeness of Discrete Distributionscited by 25
  12. 2016Differentially Private Chi-Squared Hypothesis Testing: Goodness of Fit and Independence Testingcited by 24
  13. 2018Global linear convergence of Newton's method without strong-convexity or Lipschitz gradientscited by 24
  14. 2019Privacy-preserving parametric inference: a case for robust statisticscited by 17
  15. 2020Differentially Private Bayesian Inference for Generalized Linear Modelscited by 16
  16. 2020The Cost of Privacy in Generalized Linear Models: Algorithms and Minimax Lower Boundscited by 15
  17. 2017Differentially private significance tests for regression coefficientscited by 12
  18. 2018Differentially Private Confidence Intervals for Empirical Risk Minimizationcited by 11
  19. 2014Differentially-Private Logistic Regression for Detecting Multiple-SNP Association in GWAS Databasescited by 3
  20. 2018Composite Convex Optimization with Global and Local Inexact Oraclescited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 1; 2023: 4; 2024: 11; 2025: 6; 2026: 4.

27 papers.

Sort:

By area:

  1. 2023How to DP-fy ML: A Practical Guide to Machine Learning with Differential Privacycited by 150
  2. 2023Score Attack: A Lower Bound Technique for Optimal Differentially Private Learningcited by 21
  3. 2023Faster Differentially Private Convex Optimization via Second-Order Methodscited by 13
  4. 2024Private Gradient Descent for Linear Regression: Tighter Error Bounds and Instance-Specific Uncertainty Estimationcited by 11
  5. 2024A Huber Loss Minimization Approach to Mean Estimation under User-level Differential Privacycited by 11
  6. 2022Differentially Private Image Classification from Featurescited by 7
  7. 2024BadGD: A unified data-centric framework to identify gradient descent vulnerabilitiescited by 5
  8. 2024Bounding the Excess Risk for Linear Models Trained on Marginal-Preserving, Differentially-Private, Synthetic Datacited by 3
  9. 2024Private Geometric Mediancited by 3
  10. 2024Private and Federated Stochastic Convex Optimization: Efficient Strategies for Centralized Systemscited by 3
  11. 2021Rejoinder: Gaussian Differential Privacycited by 2
  12. 2023Adaptive False Discovery Rate Control with Privacy Guaranteecited by 2
15 more
  1. 2024Resampling methods for private statistical inferencecited by 2
  2. 2024Convex Approximation of Two-Layer ReLU Networks for Hidden State Differential Privacycited by 2
  3. 2024Data Deletion for Linear Regression with Noisy SGDcited by 2
  4. 2025Tukey Depth Mechanisms for Practical Private Mean Estimationcited by 2
  5. 2025Statistical Inference for Differentially Private Stochastic Gradient Descentcited by 2
  6. 2024Uncertainty quantification by block bootstrap for differentially private stochastic gradient descentcited by 1
  7. 2024Better Locally Private Sparse Estimation Given Multiple Samples Per Usercited by 1
  8. 2025Privacy-Preserving Federated Convex Optimization: Balancing Partial-Participation and Efficiency via Noise Cancellationcited by 1
  9. 2025High-Dimensional Differentially Private Quantile Regression: Distributed Estimation and Statistical Inferencecited by 1
  10. 2025Differentially Private Hyperparameter Tuning using Local Bayesian Optimizationcited by 0
  11. 2025Non-Asymptotic Analysis of Online Local Private Learning with SGDcited by 0
  12. 2026Differentially Private Truncation of Unbounded Data via Public Second Momentscited by 0
  13. 2026Noise-Calibrated Inference from Differentially Private Sufficient Statistics in Exponential Familiescited by 0
  14. 2026DP-KFC: Data-Free Preconditioning for Privacy-Preserving Deep Learningcited by 0
  15. 2026Statistical Inference on Gradient Flowscited by 0