mlsanityData to 14 Oct 2026

Rethinking Benchmarks for Differentially Private Image Classification

arXiv 2601.17189 ↗First posted 23 Jan 2026cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

We revisit benchmarks for differentially private image classification. We suggest a comprehensive set of benchmarks, allowing researchers to evaluate techniques for differentially private machine learning in a variety of settings, including with and without additional data, in convex settings, and on a variety of qualitatively different datasets. We further test established techniques on these benchmarks in order to see which ideas remain effective in different settings. Finally, we create a publicly available leader board for the community to track progress in differentially private machine learning.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 52 of its 70 references that are in the map. First posted on arXiv 23 Jan 2026.

Builds on

52 of 70 references.

References · 52

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  5. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 7,005
  6. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,152
  7. 2016Wide Residual Networkscited by 3,070
  8. 2016Deep Learning with Differential Privacycited by 2,728
  9. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,281
  10. 2020Extracting Training Data from Large Language Modelscited by 2,189
  11. 2018Group Normalizationcited by 1,524
  12. 2019CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparisoncited by 1,423
38 more
  1. 2023Extracting Training Data from Diffusion Modelscited by 732
  2. 2022Diffusion Art or Digital Forgery? Investigating Data Replication in Diffusion Modelscited by 424
  3. 2021Large Language Models Can Be Strong Differentially Private Learnerscited by 412
  4. 2021Differentially Private Fine-tuning of Language Modelscited by 400
  5. 2021Opacus: User-Friendly Differential Privacy Library in PyTorchcited by 383
  6. 2017A Downsampled Variant of ImageNet as an Alternative to the CIFAR datasetscited by 377
  7. 2019Transfusion: Understanding Transfer Learning for Medical Imagingcited by 368
  8. 2021High-Performance Large-Scale Image Recognition Without Normalizationcited by 360
  9. 2020Large image datasets: A pyrrhic win for computer vision?cited by 237
  10. 2018Differentially Private Generative Adversarial Networkcited by 235
  11. 2020Differentially Private Learning Needs Better Features (or Much More Data)cited by 235
  12. 2022Unlocking High-Accuracy Differentially Private Image Classification through Scalecited by 224
  13. 2019R'enyi Differential Privacy of the Sampled Gaussian Mechanismcited by 204
  14. 2021Numerical Composition of Differential Privacycited by 180
  15. 2020Stability of Stochastic Gradient Descent on Nonsmooth Convex Lossescited by 173
  16. 2023How to DP-fy ML: A Practical Guide to Machine Learning with Differential Privacycited by 153
  17. 2020Tempered Sigmoid Activations for Deep Learning with Differential Privacycited by 122
  18. 2022Differentially Private Diffusion Modelscited by 119
  19. 2021Large-Scale Differentially Private BERTcited by 118
  20. 2021Learning to See by Looking at Noisecited by 92
  21. 2021Characterizing signal propagation to close the performance gap in unnormalized ResNetscited by 88
  22. 2022Position: Considerations for Differentially Private Learning with Large-Scale Public Pretrainingcited by 88
  23. 2020Large-scale Robust Deep AUC Maximization: A New Surrogate Loss and Empirical Studies on Medical Image Classificationcited by 86
  24. 2021Don't Generate Me: Training Differentially Private Generative Models with Sinkhorn Divergencecited by 54
  25. 2014Deep Roto-Translation Scattering for Object Classificationcited by 50
  26. 2019A New Analysis of Differential Privacy's Generalization Guaranteescited by 36
  27. 2022Pre-trained Perceptual Features Improve Differentially Private Image Generationcited by 28
  28. 2014Private Empirical Risk Minimization Beyond the Worst Case: The Effect of the Constraint Set Geometrycited by 24
  29. 2024How Private are DP-SGD Implementations?cited by 22
  30. 2021Not all noise is accounted equally: How differentially private learning benefits from large sampling ratescited by 21
  31. 2023Differentially Private Image Classification by Learning Priors from Random Processescited by 19
  32. 2018Replication study: Development and validation of deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographscited by 17
  33. 2023Private GANs, Revisitedcited by 17
  34. 2023Unlocking Accuracy and Fairness in Differentially Private Image Classificationcited by 16
  35. 2023LibAUC: A Deep Learning Library for X-Risk Optimizationcited by 15
  36. 2020Not one but many Tradeoffs: Privacy Vs. Utility in Differentially Private Machine Learningcited by 13
  37. 2023DPMLBench: Holistic Evaluation of Differentially Private Machine Learningcited by 13
  38. 2024Avoiding Pitfalls for Privacy Accounting of Subsampled Mechanisms under Compositioncited by 13

50 most cited of 52 All 52 (JSON)

Built on by

Not cited yet.