Fast, Sample-Efficient, Affine-Invariant Private Mean and Covariance Estimation for Subgaussian Distributions

We present a fast, differentially private algorithm for high-dimensional covariance-aware mean estimation with nearly optimal sample complexity. Only exponential-time estimators were previously known to achieve this guarantee. Given n samples from a (sub-)Gaussian distribution with unknown mean μ and covariance Sigma, our (varepsilon,δ)-differentially private estimator produces tildeμ such that |μ - tildeμ|Sigma ≤ α as long as n gtrsim tfrac d α2 + tfracd √log 1/δα varepsilon+fracdlog 1/δvarepsilon. The Mahalanobis error metric |μ - hatμ|Sigma measures the distance between hat μ and μ relative to Sigma; it characterizes the error of the sample mean. Our algorithm runs in time tildeO(ndω - 1 + nd/varepsilon), where ω < 2.38 is the matrix multiplication exponent. We adapt an exponential-time approach of Brown, Gaboardi, Smith, Ullman, and Zakynthinou (2021), giving efficient variants of stable mean and covariance estimation subroutines that also improve the sample complexity to the nearly optimal bound above. Our stable covariance estimator can be turned to private covariance estimation for unrestricted subgaussian distributions. With ngtrsim d3/2 samples, our estimate is accurate in spectral norm. This is the first such algorithm using n= o(d2) samples, answering an open question posed by Alabi et al. (2022). With ngtrsim d2 samples, our estimate is accurate in Frobenius norm. This leads to a fast, nearly optimal algorithm for private learning of unrestricted Gaussian distributions in TV distance. Duchi, Haque, and Kuditipudi (2023) obtained similar results independently and concurrently.