mlsanityData to 14 Oct 2026

Efficient Private ERM for Smooth Objectives

arXiv 1703.09947 ↗First posted 29 Mar 2017cs.LG cs.DS stat.MLcited by 61

Not on the map. Why?

Abstract (arXiv)

In this paper, we consider efficient differentially private empirical risk minimization from the viewpoint of optimization algorithms. For strongly convex and smooth objectives, we prove that gradient descent with output perturbation not only achieves nearly optimal utility, but also significantly improves the running time of previous state-of-the-art private optimization algorithms, for both ε-DP and (ε, δ)-DP. For non-convex but smooth objectives, we propose an RRPSGD (Random Round Private Stochastic Gradient Descent) algorithm, which provably converges to a stationary point with privacy guarantee. Besides the expected utility bounds, we also provide guarantees in high probability form. Experiments demonstrate that our algorithm consistently outperforms existing method in both utility and running time.

Key facts. Cited by 61 papers in the map (data through 14 Oct 2026). First posted on arXiv 29 Mar 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 18; 2022: 16; 2023: 7; 2024: 9; 2025: 7; 2026: 4.

61 papers.

Sort:

By area:

  1. 2021Deep Learning with Label Differential Privacycited by 117
  2. 2021CRFL: Certifiably Robust Federated Learning against Backdoor Attackscited by 91
  3. 2021Differential Privacy Dynamics of Langevin Diffusion and Noisy Gradient Descentcited by 73
  4. 2022Private Convex Optimization via Exponential Mechanismcited by 49
  5. 2022When Does Differentially Private Learning Not Suffer in High Dimensions?cited by 48
  6. 2022Differential Privacy and Fairness in Decisions and Learning Tasks: A Surveycited by 47
  7. 2021Differentially Private Stochastic Optimization: New Results in Convex and Non-Convex Settingscited by 45
  8. 2022SoteriaFL: A Unified Framework for Private Federated Learning with Communication Compressioncited by 34
  9. 2022Faster Rates of Convergence to Stationary Points in Differentially Private Optimizationcited by 32
  10. 2021Dynamic Differential-Privacy Preserving SGDcited by 30
  11. 2022Private Non-Convex Federated Learning Without a Trusted Servercited by 26
  12. 2023DPZero: Private Fine-Tuning of Language Models without Backpropagationcited by 26
38 more
  1. 2022Bring Your Own Algorithm for Optimal Differentially Private Stochastic Minimax Optimizationcited by 22
  2. 2021Private Non-smooth Empirical Risk Minimization and Stochastic Convex Optimization in Subquadratic Stepscited by 21
  3. 2022Beyond Uniform Lipschitz Condition in Differentially Private Optimizationcited by 19
  4. 2024Rewind-to-Delete: Certified Machine Unlearning for Nonconvex Functionscited by 19
  5. 2021Wide Network Learning with Differential Privacycited by 17
  6. 2023ReSQueing Parallel and Private Stochastic Convex Optimizationcited by 17
  7. 2022Stochastic Differentially Private and Fair Learningcited by 16
  8. 2023Faster Differentially Private Convex Optimization via Second-Order Methodscited by 13
  9. 2021Faster Rates of Private Stochastic Convex Optimizationcited by 12
  10. 2022Differentially private Riemannian optimizationcited by 12
  11. 2021A Fairness Analysis on Private Aggregation of Teacher Ensemblescited by 11
  12. 2024Taming Nonconvex Stochastic Mirror Descent with General Bregman Divergencecited by 11
  13. 2024DiSK: Differentially Private Optimizer with Simplified Kalman Filter for Noise Reductioncited by 11
  14. 2021DPlis: Boosting Utility of Differentially Private Deep Learning via Randomized Smoothingcited by 10
  15. 2022Differentially Private Regression with Unbounded Covariatescited by 10
  16. 2022Momentum Aggregation for Private Non-convex ERMcited by 10
  17. 2023DIFF2: Differential Private Optimization via Gradient Differences for Nonconvex Distributed Learningcited by 10
  18. 2024How to Make the Gradients Small Privately: Improved Rates for Differentially Private Non-Convex Optimizationcited by 10
  19. 2022Federated and Privacy-Preserving Learning of Accounting Data in Financial Statement Auditscited by 8
  20. 2024DPAdapter: Improving Differentially Private Deep Learning through Noise Tolerance Pre-trainingcited by 7
  21. 2021On the Convergence of Differentially Private Federated Learning on Non-Lipschitz Objectives, and with Normalized Client Updatescited by 5
  22. 2021Adaptive Differentially Private Empirical Risk Minimizationcited by 5
  23. 2023Differentially Private Optimization for Smooth Nonconvex ERMcited by 5
  24. 2024BadGD: A unified data-centric framework to identify gradient descent vulnerabilitiescited by 5
  25. 2021Local and Global Uniform Convexity Conditionscited by 4
  26. 2022Privacy of Autonomous Vehicles: Risks, Protection Methods, and Future Directionscited by 4
  27. 2023On the Fairness Impacts of Private Ensembles Modelscited by 4
  28. 2024Efficient Sparse Least Absolute Deviation Regression with Differential Privacycited by 4
  29. 2025An Improved Privacy and Utility Analysis of Differentially Private SGD with Bounded Domain and Smooth Lossescited by 4
  30. 2025Towards Understanding Generalization in DP-GD: A Case Study in Training Two-Layer CNNscited by 4
  31. 2022Sharper Utility Bounds for Differentially Private Modelscited by 3
  32. 2024Bounding the Excess Risk for Linear Models Trained on Marginal-Preserving, Differentially-Private, Synthetic Datacited by 3
  33. 2025Nearly Optimal Differentially Private ReLU Regressioncited by 3
  34. 2025Understanding Private Learning From Feature Perspectivecited by 3
  35. 2021Towards Sharper Utility Bounds for Differentially Private Pairwise Learningcited by 2
  36. 2021Noise-Augmented Privacy-Preserving Empirical Risk Minimization with Dual-purpose Regularizer and Privacy Budget Retrieval and Recyclingcited by 2
  37. 2024Differentially Private Random Feature Modelcited by 2
  38. 2025Privacy Amplification Through Synthetic Data: Insights from Linear Regressioncited by 2

50 most cited of 61 All 61 (JSON)