# Improving the Privacy and Practicality of Objective Perturbation for Differentially Private Linear Learners

arXiv: https://arxiv.org/abs/2401.00583 · HTML: https://mlsanity.com/paper/2401.00583/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 34 references that are in the map.
- First posted on arXiv 31 Dec 2023.

## Builds on (most cited first, 19 of 19)

- [Deep Learning with Differential Privacy](https://mlsanity.com/paper/1607.00133/) (2016)
- [Renyi Differential Privacy](https://mlsanity.com/paper/1702.07476/) (2017)
- [Privacy Amplification by Subsampling: Tight Analyses via Couplings and Divergences](https://mlsanity.com/paper/1807.01647/) (2018)
- [Subsampled R'enyi Differential Privacy and Analytical Moments Accountant](https://mlsanity.com/paper/1808.00087/) (2018)
- [Improving the Gaussian Mechanism for Differential Privacy: Analytical Calibration and Optimal Denoising](https://mlsanity.com/paper/1805.06530/) (2018)
- [R'enyi Differential Privacy of the Sampled Gaussian Mechanism](https://mlsanity.com/paper/1908.10530/) (2019)
- [Numerical Composition of Differential Privacy](https://mlsanity.com/paper/2106.02848/) (2021)
- [Practical and Private (Deep) Learning without Sampling or Shuffling](https://mlsanity.com/paper/2103.00039/) (2021)
- [Minimizing Finite Sums with the Stochastic Average Gradient](https://mlsanity.com/paper/1309.2388/) (2013)
- [The Discrete Gaussian for Differential Privacy](https://mlsanity.com/paper/2004.00010/) (2020)
- [Differentially Private Empirical Risk Minimization Revisited: Faster and More General](https://mlsanity.com/paper/1802.05251/) (2018)
- [Hyperparameter Tuning with Renyi Differential Privacy](https://mlsanity.com/paper/2110.03620/) (2021)
- [Optimal Accounting of Differential Privacy via Characteristic Function](https://mlsanity.com/paper/2106.08567/) (2021)
- [Hypothesis Testing Interpretations and Renyi Differential Privacy](https://mlsanity.com/paper/1905.09982/) (2019)
- [Private Selection from Private Candidates](https://mlsanity.com/paper/1811.07971/) (2018)
- [Tight Differential Privacy for Discrete-Valued Mechanisms and for the Subsampled Gaussian Mechanism Using FFT](https://mlsanity.com/paper/2006.07134/) (2020)
- [Dual Policy Distillation](https://mlsanity.com/paper/2006.04061/) (2020)
- [The Role of Adaptive Optimizers for Honest Private Hyperparameter Selection](https://mlsanity.com/paper/2111.04906/) (2021)
- [Privately Publishable Per-instance Privacy](https://mlsanity.com/paper/2111.02281/) (2021)

## Built on by (most cited first, 12 of 12)

- [Privacy-Preserving Instructions for Aligning Large Language Models](https://mlsanity.com/paper/2402.13659/) (2024)
- [Differentially Private Learners for Heterogeneous Treatment Effects](https://mlsanity.com/paper/2503.03486/) (2025)
- [Convex Approximation of Two-Layer ReLU Networks for Hidden State Differential Privacy](https://mlsanity.com/paper/2407.04884/) (2024)
- [Purifying Approximate Differential Privacy with Randomized Post-processing](https://mlsanity.com/paper/2503.21071/) (2025)
- [Privacy Amplification Through Synthetic Data: Insights from Linear Regression](https://mlsanity.com/paper/2506.05101/) (2025)
- [Quadratic Objective Perturbation: Curvature-Based Differential Privacy](https://mlsanity.com/paper/2605.05905/) (2026)
- [Certification-Based Differentially Private Learning](https://mlsanity.com/paper/2609.39629/) (2026)
- [Privacy-Preserving Dynamic Assortment Selection](https://mlsanity.com/paper/2410.22488/) (2024)
- [Differentially Private Two-Stage Gradient Descent for Instrumental Variable Regression](https://mlsanity.com/paper/2509.22794/) (2025)
- [Rethinking Benchmarks for Differentially Private Image Classification](https://mlsanity.com/paper/2601.17189/) (2026)
- [Random tilts to find stationary points in stochastic convex optimization](https://mlsanity.com/paper/2609.17798/) (2026)
- [Differential Privacy of Gradient Descent on Perturbed Objectives](https://mlsanity.com/paper/2610.02716/) (2026)

## Abstract (arXiv)

In the arena of privacy-preserving machine learning, differentially private stochastic gradient descent (DP-SGD) has outstripped the objective perturbation mechanism in popularity and interest. Though unrivaled in versatility, DP-SGD requires a non-trivial privacy overhead (for privately tuning the model's hyperparameters) and a computational complexity which might be extravagant for simple models such as linear and logistic regression. This paper revamps the objective perturbation mechanism with tighter privacy analyses and new computational tools that boost it to perform competitively with DP-SGD on unconstrained convex generalized linear problems.