mlsanityData to 14 Oct 2026

Private Estimation with Public Data

arXiv 2208.07984 ↗First posted 16 Aug 2022cs.LG cs.CR stat.MLcited by 32

Not on the map. Why?

Abstract (arXiv)

We initiate the study of differentially private (DP) estimation with access to a small amount of public data. For private estimation of d-dimensional Gaussians, we assume that the public data comes from a Gaussian that may have vanishing similarity in total variation distance with the underlying Gaussian of the private data. We show that under the constraints of pure or concentrated DP, d+1 public data samples are sufficient to remove any dependence on the range parameters of the private data distribution from the private sample complexity, which is known to be otherwise necessary without public data. For separated Gaussian mixtures, we assume that the underlying public and private distributions are the same, and we consider two settings: (1) when given a dimension-independent amount of public data, the private sample complexity can be improved polynomially in terms of the number of mixture components, and any dependence on the range parameters of the distribution can be removed in the approximate DP case; (2) when given an amount of public data linear in the dimension, the private sample complexity can be made independent of range parameters even under concentrated DP, and additional improvements can be made to the overall sample complexity.

Key facts. Cited by 32 papers in the map (data through 14 Oct 2026). Builds on 28 of its 47 references that are in the map. First posted on arXiv 16 Aug 2022.

Builds on

28 of 47 references.

References · 28

  1. 2021Large Language Models Can Be Strong Differentially Private Learnerscited by 412
  2. 2021Differentially Private Fine-tuning of Language Modelscited by 400
  3. 2016Semi-supervised Knowledge Transfer for Deep Learning from Private Training Datacited by 348
  4. 2018Scalable Private Learning with PATEcited by 269
  5. 2020Differentially Private Learning Needs Better Features (or Much More Data)cited by 235
  6. 2016Robust Estimators in High Dimensions without the Computational Intractabilitycited by 209
  7. 2018Privately Learning High-Dimensional Distributionscited by 104
  8. 2020Bypassing the Ambient Dimension: Private SGD with Gradient Subspace Identificationcited by 90
  9. 2021Do Not Let Privacy Overbill Utility: Gradient Embedding Perturbation for Private Learningcited by 86
  10. 2020CoinPress: Practical Private Mean and Covariance Estimationcited by 74
  11. 2021Differential privacy and robust statistics in high dimensionscited by 64
  12. 2014Private Learning and Sanitization: Pure vs. Approximate Differential Privacycited by 63
16 more
  1. 2019Private Hypothesis Selectioncited by 60
  2. 2021Efficient Mean Estimation with Pure Differential Privacy via a Sum-of-Squares Exponential Mechanismcited by 50
  3. 2021Public Data-Assisted Mirror Descent for Private Model Trainingcited by 49
  4. 2020On the Sample Complexity of Privately Learning Unbounded High-Dimensional Gaussianscited by 41
  5. 2019Limits of Private Learning with Access to Public Datacited by 39
  6. 2021Leveraging Public Data for Practical Private Query Releasecited by 39
  7. 2021Private and polynomial time algorithms for learning Gaussians and beyondcited by 39
  8. 2021Private Robust Estimation by Stabilizing Convex Relaxationscited by 38
  9. 2015Simultaneous Private Learning of Multiple Conceptscited by 34
  10. 2021A Private and Computationally-Efficient Estimator for Unbounded Gaussianscited by 34
  11. 2019Differentially Private Algorithms for Learning Mixtures of Separated Gaussianscited by 27
  12. 2020Private Query Release Assisted by Public Datacited by 27
  13. 2020Learning from Mixtures of Private and Public Populationscited by 19
  14. 2019Privately Answering Classification Queries in the Agnostic PAC Modelcited by 18
  15. 2017BLENDER: Enabling Local Search with a Hybrid Differential Privacy Modelcited by 14
  16. 2018The Power of The Hybrid Model for Mean Estimationcited by 11

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 1; 2023: 12; 2024: 9; 2025: 6; 2026: 4.

32 papers.

Sort:

By area:

  1. 2023Why Is Public Pretraining Necessary for Private Model Training?cited by 37
  2. 2023Selective Pre-training for Private Fine-tuningcited by 23
  3. 2023Polynomial Time and Private Learning of Unbounded Gaussian Mixture Modelscited by 19
  4. 2023Private Distribution Learning with Public Data: The View from Sample Compressioncited by 14
  5. 2023A Bias-Accuracy-Privacy Trilemma for Statistical Estimationcited by 12
  6. 2025Stochastic Forward-Backward Deconvolution: Training Diffusion Models with Finite Noisy Datasetscited by 11
  7. 2023Optimal Differentially Private Model Training with Public Datacited by 9
  8. 2025Synthetic Tabular Data: Methods, Attacks and Defensescited by 9
  9. 2023Mean Estimation Under Heterogeneous Privacy: Some Privacy Can Be Freecited by 6
  10. 2023Mean Estimation Under Heterogeneous Privacy Demandscited by 6
  11. 2024Oracle-Efficient Differentially Private Learning with Public Datacited by 6
  12. 2024Public-data Assisted Private Stochastic Optimization: Power and Limitationscited by 6
20 more
  1. 2022Learning-Augmented Private Algorithms for Multiple Quantile Releasecited by 5
  2. 2023A Polynomial Time, Pure Differentially Private Estimator for Binary Product Distributionscited by 5
  3. 2023Mixtures of Gaussians are Privately Learnable with a Polynomial Number of Samplescited by 5
  4. 2023Sample-Optimal Locally Private Hypothesis Selection and the Provable Benefits of Interactivitycited by 3
  5. 2024Not All Learnable Distribution Classes are Privately Learnablecited by 3
  6. 2024Credit Attribution and Stable Compressioncited by 3
  7. 2024Locally Private Sampling with Public Datacited by 3
  8. 2026On the Price of Privacy for Language Identification and Generationcited by 3
  9. 2024Agnostic Private Density Estimation for GMMs via List Global Stabilitycited by 2
  10. 2024Private Estimation when Data and Privacy Demands are Correlatedcited by 2
  11. 2024Differentially Private Multi-Sampling from Distributionscited by 2
  12. 2025Leveraging Vertical Public-Private Split for Improved Synthetic Data Generationcited by 2
  13. 2025Optimal Differentially Private Sampling of Unbounded Gaussianscited by 1
  14. 2023Differentially Private Domain Adaptation with Theoretical Guaranteescited by 0
  15. 2024Sample-Efficient Private Learning of Mixtures of Gaussianscited by 0
  16. 2025Lower Bounds for Public-Private Learning under Distribution Shiftcited by 0
  17. 2025Enhancing Differentially Private Linear Regression via Public Second-Momentcited by 0
  18. 2026Differentially Private Truncation of Unbounded Data via Public Second Momentscited by 0
  19. 2026Limits of Personalizing Differential Privacy Budgetscited by 0
  20. 2026High-Dimensional Asymptotics and Dataset Selection for Private Transfer Learningcited by 0