mlsanityData to 14 Oct 2026

Privately Learning Subspaces

arXiv 2106.00001 ↗First posted 28 May 2021cs.CR cs.DS cs.LGcited by 16

Not on the map. Why?

Abstract (arXiv)

Private data analysis suffers a costly curse of dimensionality. However, the data often has an underlying low-dimensional structure. For example, when optimizing via gradient descent, the gradients often lie in or near a low-dimensional subspace. If that low-dimensional structure can be identified, then we can avoid paying (in terms of privacy or accuracy) for the high ambient dimension. We present differentially private algorithms that take input data sampled from a low-dimensional linear subspace (possibly with a small amount of error) and output that subspace (or an approximation to it). These algorithms can serve as a pre-processing step for other procedures.

Key facts. Cited by 16 papers in the map (data through 14 Oct 2026). Builds on 17 of its 49 references that are in the map. First posted on arXiv 28 May 2021.

Builds on

17 of 49 references.

References · 17

  1. 2016Deep Learning with Differential Privacycited by 2,728
  2. 2017Visualizing the Loss Landscape of Neural Netscited by 1,130
  3. 2018Measuring the Intrinsic Dimension of Objective Landscapescited by 370
  4. 2016Concentrated Differential Privacy: Simplifications, Extensions, and Lower Boundscited by 346
  5. 2018Gradient Descent Happens in a Tiny Subspacecited by 217
  6. 2020Bypassing the Ambient Dimension: Private SGD with Gradient Subspace Identificationcited by 90
  7. 2013Fingerprinting Codes and the Price of Approximate Differential Privacycited by 82
  8. 2018Private Selection from Private Candidatescited by 75
  9. 2013The Noisy Power Method: A Meta Algorithm with Applicationscited by 58
  10. 2015Between Pure and Approximate Differential Privacycited by 56
  11. 2016Rate-Optimal Perturbation Bounds for Singular Subspaces with Applications to High-Dimensional Statisticscited by 45
  12. 2019Hessian based analysis of SGD for Deep Nets: Dynamics and Generalizationcited by 37
5 more
  1. 2015Simultaneous Private Learning of Multiple Conceptscited by 34
  2. 2020Private Query Release Assisted by Public Datacited by 27
  3. 2014Private Multiplicative Weights Beyond Linear Queriescited by 17
  4. 2018Towards Instance-Optimal Private Query Releasecited by 9
  5. 2020How neural networks find generalizable solutions: Self-tuned annealing in deep learningcited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 3; 2022: 2; 2023: 4; 2024: 4; 2025: 2; 2026: 1.

16 papers.

Sort:

  1. 2021Private and polynomial time algorithms for learning Gaussians and beyondcited by 39
  2. 2024Does SGD really happen in tiny subspaces?cited by 36
  3. 2021FriendlyCore: Practical Differentially Private Aggregationcited by 34
  4. 2021A Private and Computationally-Efficient Estimator for Unbounded Gaussianscited by 34
  5. 2023Fast, Sample-Efficient, Affine-Invariant Private Mean and Covariance Estimation for Subgaussian Distributionscited by 22
  6. 2022Differentially Private Covariance Revisitedcited by 15
  7. 2024Efficiently Computing Similarities to Private Datasetscited by 9
  8. 2023Smooth Lower Bounds for Differentially Private Algorithms via Padding-and-Permuting Fingerprinting Codescited by 8
  9. 2023PLAN: Variance-Aware Private Mean Estimationcited by 6
  10. 2024Dimension-free Private Mean Estimation for Anisotropic Distributionscited by 6
  11. 2023Differentially Private Low-dimensional Synthetic Data from High-dimensional Datasetscited by 4
  12. 2025An Iterative Algorithm for Differentially Private k-PCA with Adaptive Noisecited by 3
4 more
  1. 2024On Differentially Private Subspace Estimation in a Distribution-Free Settingcited by 1
  2. 2022Privately Estimating a Gaussian: Efficient, Robust and Optimalcited by 0
  3. 2025Towards Reliable and Generalizable Differentially Private Machine Learning (Extended Version)cited by 0
  4. 2026Privacy-preserving federated tensor decomposition of single-cell immune data: recovering multicellular programs across institutionscited by 0