mlsanityData to 7 Oct 2026

Improving the Privacy and Practicality of Objective Perturbation for Differentially Private Linear Learners

arXiv 2401.00583 ↗First posted 31 Dec 2023cs.LG cs.CRcited by 11

Not on the map. Why?

Abstract (arXiv)

In the arena of privacy-preserving machine learning, differentially private stochastic gradient descent (DP-SGD) has outstripped the objective perturbation mechanism in popularity and interest. Though unrivaled in versatility, DP-SGD requires a non-trivial privacy overhead (for privately tuning the model's hyperparameters) and a computational complexity which might be extravagant for simple models such as linear and logistic regression. This paper revamps the objective perturbation mechanism with tighter privacy analyses and new computational tools that boost it to perform competitively with DP-SGD on unconstrained convex generalized linear problems.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 19 of its 34 references that are in the map. First posted on arXiv 31 Dec 2023.

Builds on

19 of 34 references.

References · 19

  1. 2016Deep Learning with Differential Privacycited by 2,682
  2. 2017Renyi Differential Privacycited by 643
  3. 2018Privacy Amplification by Subsampling: Tight Analyses via Couplings and Divergencescited by 237
  4. 2018Subsampled R'enyi Differential Privacy and Analytical Moments Accountantcited by 234
  5. 2018Improving the Gaussian Mechanism for Differential Privacy: Analytical Calibration and Optimal Denoisingcited by 216
  6. 2019R'enyi Differential Privacy of the Sampled Gaussian Mechanismcited by 199
  7. 2021Numerical Composition of Differential Privacycited by 174
  8. 2021Practical and Private (Deep) Learning without Sampling or Shufflingcited by 169
  9. 2013Minimizing Finite Sums with the Stochastic Average Gradientcited by 164
  10. 2020The Discrete Gaussian for Differential Privacycited by 134
  11. 2018Differentially Private Empirical Risk Minimization Revisited: Faster and More Generalcited by 127
  12. 2021Hyperparameter Tuning with Renyi Differential Privacycited by 104
7 more
  1. 2019Hypothesis Testing Interpretations and Renyi Differential Privacycited by 84
  2. 2021Optimal Accounting of Differential Privacy via Characteristic Functioncited by 83
  3. 2018Private Selection from Private Candidatescited by 75
  4. 2020Tight Differential Privacy for Discrete-Valued Mechanisms and for the Subsampled Gaussian Mechanism Using FFTcited by 43
  5. 2020Dual Policy Distillationcited by 25
  6. 2021The Role of Adaptive Optimizers for Honest Private Hyperparameter Selectioncited by 25
  7. 2021Privately Publishable Per-instance Privacycited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 4; 2026: 4.

11 papers.

  1. 2024Privacy-Preserving Instructions for Aligning Large Language Modelscited by 34
  2. 2025Differentially Private Learners for Heterogeneous Treatment Effectscited by 5
  3. 2024Convex Approximation of Two-Layer ReLU Networks for Hidden State Differential Privacycited by 2
  4. 2025Purifying Approximate Differential Privacy with Randomized Post-processingcited by 2
  5. 2025Privacy Amplification Through Synthetic Data: Insights from Linear Regressioncited by 2
  6. 2024Privacy-Preserving Dynamic Assortment Selectioncited by 0
  7. 2025Differentially Private Two-Stage Gradient Descent for Instrumental Variable Regressioncited by 0
  8. 2026Rethinking Benchmarks for Differentially Private Image Classificationcited by 0
  9. 2026Quadratic Objective Perturbation: Curvature-Based Differential Privacycited by 0
  10. 2026Random tilts to find stationary points in stochastic convex optimizationcited by 0
  11. 2026Certification-Based Differentially Private Learningcited by 0