mlsanityData to 14 Oct 2026

Optimal Differentially Private PCA and Estimation for Spiked Covariance Matrices

arXiv 2401.03820 ↗First posted 8 Jan 2024math.ST cs.IT math.ITcited by 11

Not on the map. Why?

Abstract (arXiv)

Estimating a covariance matrix and its associated principal components is a fundamental problem in contemporary statistics. While optimal estimation procedures have been developed with well-understood properties, the increasing demand for privacy preservation introduces new complexities to this classical problem. In this paper, we study optimal differentially private Principal Component Analysis (PCA) and covariance estimation within the spiked covariance model. We precisely characterize the sensitivity of eigenvalues and eigenvectors under this model and establish the minimax rates of convergence for estimating both the principal components and covariance matrix. These rates hold up to logarithmic factors and encompass general Schatten norms, including spectral norm, Frobenius norm, and nuclear norm as special cases. We propose computationally efficient differentially private estimators and prove their minimax optimality for sub-Gaussian distributions, up to logarithmic factors. Additionally, matching minimax lower bounds are established. Notably, compared to the existing literature, our results accommodate a diverging rank, a broader range of signal strengths, and remain valid even when the sample size is much smaller than the dimension, provided the signal strength is sufficiently strong. Both simulation studies and real data experiments demonstrate the merits of our method.

Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Builds on 17 of its 64 references that are in the map. First posted on arXiv 8 Jan 2024.

Builds on

17 of 64 references.

References · 17

  1. 2014RAPPOR: Randomized Aggregatable Privacy-Preserving Ordinal Responsecited by 272
  2. 2017Collecting Telemetry Data Privatelycited by 149
  3. 2016Minimax Optimal Procedures for Locally Private Estimationcited by 107
  4. 2018Privately Learning High-Dimensional Distributionscited by 104
  5. 2020Differentially Private Assouad, Fano, and Le Camcited by 55
  6. 2017Tensor SVD: Statistical and Computational Limitscited by 53
  7. 2021Covariance-Aware Private Mean Estimation Without Private Covariance Estimationcited by 46
  8. 2018Heteroskedastic PCA: Algorithm, Optimality, and Applicationscited by 34
  9. 2022DP-PCA: Statistically Optimal and Differentially Private PCAcited by 24
  10. 2023Score Attack: A Lower Bound Technique for Optimal Differentially Private Learningcited by 21
  11. 2023Better and Simpler Lower Bounds for Differentially Private Statistical Estimationcited by 17
  12. 2020Estimation of the number of spiked eigenvalues in a covariance matrix by bulk eigenvalue matching analysiscited by 15
5 more
  1. 2022Differentially Private Covariance Revisitedcited by 15
  2. 2021Private Alternating Least Squares: Practical Private Matrix Completion with Tighter Ratescited by 10
  3. 2022Private Query Release via the Johnson-Lindenstrauss Transformcited by 8
  4. 2022Re-Analyze Gauss: Bounds for Private Matrix Approximation via Dyson Brownian Motioncited by 8
  5. 2015Optimal Estimation of Low Rank Density Matricescited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 4; 2025: 2; 2026: 5.

11 papers.

  1. 2024Federated Nonparametric Hypothesis Testing with Differential Privacy Constraints: Optimal Rates and Adaptive Testscited by 6
  2. 2024Federated PCA and Estimation for Spiked Covariance Matrices: Optimal Rates and Efficient Algorithmcited by 3
  3. 2025An Iterative Algorithm for Differentially Private k-PCA with Adaptive Noisecited by 3
  4. 2026Minimax and Adaptive Covariance Matrix Estimation under Differential Privacycited by 1
  5. 2026On the Curse of Dimensionality in Private Sparse Covariance Estimation and PCAcited by 1
  6. 2026Gap-Free Streaming PCA Beyond Rank-One Updates: Near-Optimal Rates and Applications to Differential Privacycited by 1
  7. 2024Near-Optimal differentially private low-rank trace regression with guaranteed private initializationcited by 0
  8. 2024Distributed quasi-Newton robust estimation under differential privacycited by 0
  9. 2025Subspace Recovery in Winsorized PCA: Insights into Accuracy and Robustnesscited by 0
  10. 2026Differentially Private Manifold Denoisingcited by 0
  11. 2026Enhancing Differentially Private Mechanisms via Empirical Bayescited by 0