mlsanityData to 7 Oct 2026

Towards Understanding the Impact of Model Size on Differential Private Classification

arXiv 2111.13895 ↗First posted 27 Nov 2021cs.LG cs.CRcited by 10

Not on the map. Why?

Abstract (arXiv)

Differential privacy (DP) is an essential technique for privacy-preserving. It was found that a large model trained for privacy preserving performs worse than a smaller model (e.g. ResNet50 performs worse than ResNet18). To better understand this phenomenon, we study high dimensional DP learning from the viewpoint of generalization. Theoretically, we show that for the simple Gaussian model with even small DP noise, if the dimension is large enough, then the classification error can be as bad as the random guessing. Then we propose a feature selection method to reduce the size of the model, based on a new metric which trades off the classification accuracy and privacy preserving. Experiments on real data support our theoretical results and demonstrate the advantage of the proposed method.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 8 of its 25 references that are in the map. First posted on arXiv 27 Nov 2021.

Builds on

8 of 25 references.

References · 8

  1. 2019Adversarial Examples Are Not Bugs, They Are Featurescited by 950
  2. 2018Adversarially Robust Generalization Requires More Datacited by 318
  3. 2020Differentially Private Learning Needs Better Features (or Much More Data)cited by 233
  4. 2020Understanding Gradient Clipping in Private SGD: A Geometric Perspectivecited by 140
  5. 2014Differentially Private Empirical Risk Minimization: Efficient Algorithms and Tight Error Boundscited by 104
  6. 2020More Data Can Expand the Generalization Gap Between Adversarially Robust and Standard Modelscited by 27
  7. 2021An Efficient DP-SGD Mechanism for Large Scale NLP Modelscited by 27
  8. 2014Sparsifying the Fisher Linear Discriminant by Rotationcited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 3; 2024: 1; 2025: 2; 2026: 2.

10 papers.

  1. 2022Unlocking High-Accuracy Differentially Private Image Classification through Scalecited by 218
  2. 2022Fine-Tuning with Differential Privacy Necessitates an Additional Hyperparameter Searchcited by 15
  3. 2023PrivImage: Differentially Private Synthetic Image Generation using Diffusion Models with Semantic-Aware Pretrainingcited by 12
  4. 2023SoK: Unintended Interactions among Machine Learning Defenses and Riskscited by 6
  5. 2026Fundamental Limitations of Favorable Privacy-Utility Guarantees for DP-SGDcited by 4
  6. 2025An Optimization Framework for Differentially Private Sparse Fine-Tuningcited by 2
  7. 2023DP-DCAN: Differentially Private Deep Contrastive Autoencoder Network for Single-cell Clusteringcited by 1
  8. 2024Federating Dynamic Models using Early-Exit Architectures for Automatic Speech Recognition on Heterogeneous Clientscited by 1
  9. 2025Differentially Private 2D Human Pose Estimationcited by 0
  10. 2026PACZero: PAC-Private Fine-Tuning of Language Models via Sign Quantizationcited by 0