Differentially Private Non-convex Distributionally Robust Optimization
Not on the map. Why?
Abstract (arXiv)
Real-world deployments routinely face distribution shifts, group imbalances, and adversarial perturbations, under which the traditional Empirical Risk Minimization (ERM) framework can degrade severely. Distributionally Robust Optimization (DRO) addresses this issue by optimizing the worst-case expected loss over an uncertainty set of distributions, offering a principled approach to robustness. Meanwhile, as training data in DRO always involves sensitive information, safeguarding it against leakage under Differential Privacy (DP) is essential. In contrast to classical DP-ERM, DP-DRO has received much less attention due to its minimax optimization structure with uncertainty constraint. To bridge the gap, we provide a comprehensive study of DP-(finite-sum)-DRO with psi-divergence and non-convex loss. First, we study DRO with general psi-divergence by reformulating it as a minimization problem, and develop a novel (varepsilon, δ)-DP optimization method, called DP Double-Spider, tailored to this structure. Under mild assumptions, we show that it achieves a utility bound of O(frac1√n+ (frac√d log (1/δ)n varepsilon)2/3) in terms of the gradient norm, where n denotes the data size and d denotes the model dimension. We further improve the utility rate for specific divergences. In particular, for DP-DRO with KL-divergence, by transforming the problem into a compositional finite-sum optimization problem, we develop a DP Recursive-Spider method and show that it achieves a utility bound of O((frac√d log(1/δ)nvarepsilon)2/3 ), matching the best-known result for non-convex DP-ERM. Experimentally, we demonstrate that our proposed methods outperform existing approaches for DP minimax optimization.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 42 of its 60 references that are in the map. First posted on arXiv 18 Feb 2026.
Builds on
42 of 60 references.
References · 42
- 2014Explaining and Harnessing Adversarial Examplescited by 7,287
- 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
- 2016Deep Learning with Differential Privacycited by 2,682
- 2019Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalizationcited by 1,232
- 2019Why gradient clipping accelerates training: A theoretical justification for adaptivitycited by 335
- 2018Learning Models with Uniform Performance via Distributionally Robust Optimizationcited by 334
- 2017Certifying Some Distributional Robustness with Principled Adversarial Trainingcited by 333
- 2019Momentum-Based Variance Reduction in Non-Convex SGDcited by 302
- 2018SPIDER: Near-Optimal Non-Convex Optimization via Stochastic Path Integrated Differential Estimatorcited by 283
- 2019Private Stochastic Convex Optimization with Optimal Ratescited by 183
- 2016Statistics of Robust Optimization: A Generalized Empirical Likelihood Approachcited by 182
- 2020Large-Scale Methods for Distributionally Robust Optimizationcited by 179
30 more
- 2016Variance-based regularization with convex objectivescited by 153
- 2018Differentially Private Empirical Risk Minimization Revisited: Faster and More Generalcited by 127
- 2018Weakly-Convex Concave Min-Max Optimization: Provable Algorithms and Applications in Machine Learningcited by 62
- 2020On Differentially Private Stochastic Convex Optimization with Heavy-tailed Datacited by 51
- 2020The limits of min-max optimization algorithms: convergence to spurious non-critical setscited by 50
- 2021Differentially Private Stochastic Optimization: New Results in Convex and Non-Convex Settingscited by 45
- 2020An Online Method for A Class of Distributionally Robust Optimization with Non-Convex Objectivescited by 42
- 2021High Dimensional Differentially Private Stochastic Optimization with Heavy-tailed Datacited by 42
- 2021Non-convex Distributionally Robust Optimization: Non-asymptotic Analysiscited by 39
- 2023Generalized-Smooth Nonconvex Optimization is As Efficient As Smooth Nonconvex Optimizationcited by 35
- 2022Faster Rates of Convergence to Stationary Points in Differentially Private Optimizationcited by 32
- 2020Private Stochastic Non-Convex Optimization: Adaptive Algorithms and Tighter Generalization Boundscited by 26
- 2022Differentially Private SGDA for Minimax Problemscited by 23
- 2022Bring Your Own Algorithm for Optimal Differentially Private Stochastic Minimax Optimizationcited by 22
- 2023Practical Differentially Private and Byzantine-resilient Federated Learningcited by 18
- 2022Differentially Private Generalized Linear Models Revisitedcited by 17
- 2022Stochastic Constrained DRO with a Complexity Independent of Sample Sizecited by 16
- 2020Empirical Risk Minimization in the Non-interactive Local Model of Differential Privacycited by 15
- 2019Non-asymptotic Analysis of Stochastic Methods for Non-Smooth Non-Convex Regularized Problemscited by 14
- 2025Short-length Adversarial Training Helps LLMs Defend Long-length Jailbreak Attacks: Theoretical and Empirical Evidencecited by 13
- 2025Towards User-level Private Reinforcement Learning with Human Feedbackcited by 13
- 2024Differentially Private Worst-group Risk Minimizationcited by 9
- 2023Differentially Private Stochastic Convex Optimization in (Non)-Euclidean Space Revisitedcited by 8
- 2023Theoretical Analysis of Robust Overfitting for Wide DNNs: An NTK Approachcited by 8
- 2022Differentially Private Temporal Difference Learning with Stochastic Nonconvex-Strongly-Concave Optimizationcited by 7
- 2025Improved Rates of Differentially Private Nonconvex-Strongly-Concave Minimax Optimizationcited by 5
- 2024Large-Scale Non-convex Stochastic Constrained Distributionally Robust Optimizationcited by 3
- 2025Second-Order Convergence in Private Stochastic Non-Convex Optimizationcited by 3
- 2023Differential Privacy via Distributionally Robust Optimizationcited by 2
- 2025Beyond Ordinary Lipschitz Constraints: Differentially Private Stochastic Optimization with Tsybakov Noise Conditioncited by 2
Built on by
Citing papers per year: 2026: 1.
1 paper.