# DP-LSSGD: A Stochastic Optimization Method to Lift the Utility in Privacy-Preserving ERM

arXiv: https://arxiv.org/abs/1906.12056 · HTML: https://mlsanity.com/paper/1906.12056/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Jun 2019.

## Built on by (most cited first, 15 of 15)

- [Selective Differential Privacy for Language Modeling](https://mlsanity.com/paper/2108.12944/) (2021)
- [Survey: Leakage and Privacy at Inference Time](https://mlsanity.com/paper/2107.01614/) (2021)
- [Split-and-Denoise: Protect large language model inference with local differential privacy](https://mlsanity.com/paper/2310.09130/) (2023)
- [DiSK: Differentially Private Optimizer with Simplified Kalman Filter for Noise Reduction](https://mlsanity.com/paper/2410.03883/) (2024)
- [DPlis: Boosting Utility of Differentially Private Deep Learning via Randomized Smoothing](https://mlsanity.com/paper/2103.01496/) (2021)
- [DIFF2: Differential Private Optimization via Gradient Differences for Nonconvex Distributed Learning](https://mlsanity.com/paper/2302.03884/) (2023)
- [DP-FedPGN: Finding Global Flat Minima for Differentially Private Federated Learning via Penalizing Gradient Norm](https://mlsanity.com/paper/2510.27504/) (2025)
- [DOPPLER: Differentially Private Optimizers with Low-pass Filter for Privacy Noise Reduction](https://mlsanity.com/paper/2408.13460/) (2024)
- [Adaptive Differentially Private Empirical Risk Minimization](https://mlsanity.com/paper/2110.07435/) (2021)
- [On the Fairness Impacts of Private Ensembles Models](https://mlsanity.com/paper/2305.11807/) (2023)
- [Differentially Private 2D Human Pose Estimation](https://mlsanity.com/paper/2504.10190/) (2025)
- [Adaptive Selection of LoRA Components in Privacy-Preserving Federated Learning](https://mlsanity.com/paper/2605.05769/) (2026)
- [Applying Differential Privacy to Tensor Completion](https://mlsanity.com/paper/2110.00539/) (2021)
- [Deep Learning with Data Privacy via Residual Perturbation](https://mlsanity.com/paper/2408.05723/) (2024)
- [The Fair Game: Auditing & Debiasing AI Algorithms Over Time](https://mlsanity.com/paper/2508.06443/) (2025)

## Abstract (arXiv)

Machine learning (ML) models trained by differentially private stochastic gradient descent (DP-SGD) have much lower utility than the non-private ones. To mitigate this degradation, we propose a DP Laplacian smoothing SGD (DP-LSSGD) to train ML models with differential privacy (DP) guarantees. At the core of DP-LSSGD is the Laplacian smoothing, which smooths out the Gaussian noise used in the Gaussian mechanism. Under the same amount of noise used in the Gaussian mechanism, DP-LSSGD attains the same DP guarantee, but in practice, DP-LSSGD makes training both convex and nonconvex ML models more stable and enables the trained models to generalize better. The proposed algorithm is simple to implement and the extra computational complexity and memory overhead compared with DP-SGD are negligible. DP-LSSGD is applicable to train a large variety of ML models, including DNNs. The code is available at urlhttps://github.com/BaoWangMath/DP-LSSGD.