Faster Rates of Private Stochastic Convex Optimization

In this paper, we revisit the problem of Differentially Private Stochastic Convex Optimization (DP-SCO) and provide excess population risks for some special classes of functions that are faster than the previous results of general convex and strongly convex functions. In the first part of the paper, we study the case where the population risk function satisfies the Tysbakov Noise Condition (TNC) with some parameter θ>1. Specifically, we first show that under some mild assumptions on the loss functions, there is an algorithm whose output could achieve an upper bound of tildeO((frac1√n+frac√dlog frac1δnε)^fracθθ-1) for (ε, δ)-DP when θ≥ 2, here n is the sample size and d is the dimension of the space. Then we address the inefficiency issue, improve the upper bounds by Poly(log n) factors and extend to the case where θ≥ barθ>1 for some known barθ. Next we show that the excess population risk of population functions satisfying TNC with parameter θ≥ 2 is always lower bounded by Omega((fracdnε)^fracθθ-1) and Omega((frac√dlog frac1δnε)^fracθθ-1) for ε-DP and (ε, δ)-DP, respectively. In the second part, we focus on a special case where the population risk function is strongly convex. Unlike the previous studies, here we assume the loss function is em non-negative and em the optimal value of population risk is sufficiently small. With these additional assumptions, we propose a new method whose output could achieve an upper bound of O(fracdlogfrac1δn2ε2+frac1nτ) for any τ≥ 1 in (ε,δ)-DP model if the sample size n is sufficiently large.