mlsanityData to 7 Oct 2026

Private Set Generation with Discriminative Information

arXiv 2211.04446 ↗First posted 7 Nov 2022cs.CR cs.AI cs.LGcited by 34

Not on the map. Why?

Abstract (arXiv)

Differentially private data generation techniques have become a promising solution to the data privacy challenge -- it enables sharing of data while complying with rigorous privacy guarantees, which is essential for scientific progress in sensitive domains. Unfortunately, restricted by the inherent complexity of modeling high-dimensional distributions, existing private generative models are struggling with the utility of synthetic samples. In contrast to existing works that aim at fitting the complete data distribution, we directly optimize for a small set of samples that are representative of the distribution under the supervision of discriminative information from downstream tasks, which is generally an easier task and more suitable for private training. Our work provides an alternative view for differentially private generation of high-dimensional data and introduces a simple yet effective method that greatly improves the sample utility of state-of-the-art approaches.

Key facts. Cited by 34 papers in the map (data through 7 Oct 2026). Builds on 32 of its 53 references that are in the map. First posted on arXiv 7 Nov 2022.

Builds on

32 of 53 references.

References · 32

  1. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
  2. 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,590
  3. 2016Deep Learning with Differential Privacycited by 2,682
  4. 2016Learning without Forgettingcited by 2,465
  5. 2016iCaRL: Incremental Classifier and Representation Learningcited by 2,313
  6. 2015Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networkscited by 2,048
  7. 2016Instance Normalization: The Missing Ingredient for Fast Stylizationcited by 1,187
  8. 2020Training Generative Adversarial Networks with Limited Datacited by 1,184
  9. 2017Deep Image Priorcited by 1,097
  10. 2018An Empirical Study of Example Forgetting during Deep Neural Network Learningcited by 669
  11. 2017Renyi Differential Privacycited by 643
  12. 2018Progress & Compress: A scalable framework for continual learningcited by 516
20 more
  1. 2020Dataset Condensation with Gradient Matchingcited by 510
  2. 2021Opacus: User-Friendly Differential Privacy Library in PyTorchcited by 375
  3. 2021Dataset Condensation with Distribution Matchingcited by 362
  4. 2016Semi-supervised Knowledge Transfer for Deep Learning from Private Training Datacited by 347
  5. 2021Dataset Condensation with Differentiable Siamese Augmentationcited by 290
  6. 2018Scalable Private Learning with PATEcited by 266
  7. 2018Differentially Private Generative Adversarial Networkcited by 235
  8. 2018Subsampled R'enyi Differential Privacy and Analytical Moments Accountantcited by 234
  9. 2020Differentially Private Learning Needs Better Features (or Much More Data)cited by 233
  10. 2018Dataset Distillationcited by 224
  11. 2020Mnemonics Training: Multi-Class Incremental Learning without Forgettingcited by 220
  12. 2019R'enyi Differential Privacy of the Sampled Gaussian Mechanismcited by 199
  13. 2022Privacy for Free: How does Dataset Condensation Help Privacy?cited by 119
  14. 2020DP-CGAN: Differentially Private Synthetic Data and Label Generationcited by 118
  15. 2020GS-WGAN: A Gradient-Sanitized Approach for Learning Differentially Private Generatorscited by 117
  16. 2020DP-MERF: Differentially Private Mean Embeddings with Random Features for Practical Privacy-Preserving Data Generationcited by 78
  17. 2021Don't Generate Me: Training Differentially Private Generative Models with Sinkhorn Divergencecited by 54
  18. 2019G-PATE: Scalable Differentially Private Data Generator via Private Aggregation of Teacher Discriminatorscited by 53
  19. 2020New Oracle-Efficient Algorithms for Private Synthetic Data Releasecited by 43
  20. 2021DataLens: Scalable Privacy Preserving Training via Gradient Compression and Aggregationcited by 31

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 17; 2024: 6; 2025: 7; 2026: 4.

34 papers.

Sort:

By area:

  1. 2023Dataset Distillation: A Comprehensive Reviewcited by 126
  2. 2023Towards Lossless Dataset Distillation via Difficulty-Aligned Trajectory Matchingcited by 118
  3. 2023A Comprehensive Survey of Dataset Distillationcited by 114
  4. 2023Data Distillation: A Surveycited by 86
  5. 2023Differentially Private Diffusion Models Generate Useful Synthetic Imagescited by 85
  6. 2025Dataset Distillation with Neural Characteristic Function: A Minmax Perspectivecited by 45
  7. 2023A Survey on Dataset Distillation: Approaches, Applications and Future Directionscited by 32
  8. 2024DiLM: Distilling Dataset into Language Model for Text-level Dataset Distillationcited by 20
  9. 2024Importance-Aware Adaptive Dataset Distillationcited by 18
  10. 2023Private GANs, Revisitedcited by 17
  11. 2023Differentially Private Neural Tangent Kernels for Privacy-Preserving Data Generationcited by 15
  12. 2025Hyperbolic Dataset Distillationcited by 15
22 more
  1. 2024Not All Samples Should Be Utilized Equally: Towards Understanding and Improving Dataset Distillationcited by 14
  2. 2023DPMLBench: Holistic Evaluation of Differentially Private Machine Learningcited by 13
  3. 2023A Comprehensive Study on Dataset Distillation: Performance, Privacy, Robustness and Fairnesscited by 9
  4. 2023FedLAP-DP: Federated Learning by Sharing Differentially Private Loss Approximationscited by 4
  5. 2023A Unified View of Differentially Private Deep Generative Modelingcited by 4
  6. 2025Improving Noise Efficiency in Privacy-preserving Dataset Distillationcited by 4
  7. 2023On the Size and Approximation Error of Distilled Setscited by 3
  8. 2023Boosting the Cross-Architecture Generalization of Dataset Distillation through an Empirical Studycited by 3
  9. 2024Towards Biologically Plausible and Private Gene Expression Data Generationcited by 2
  10. 2024Learning Differentially Private Diffusion Models via Stochastic Adversarial Distillationcited by 2
  11. 2023Differentially Private Kernel Inducing Points using features from ScatterNets (DP-KIP-ScatterNet) for Privacy Preserving Data Distillationcited by 1
  12. 2023AST: Effective Dataset Distillation through Alignment with Smooth and High-Quality Expert Trajectoriescited by 1
  13. 2024One-Shot Collaborative Data Distillationcited by 1
  14. 2025Laplace Sample Information: Data Informativeness Through a Bayesian Lenscited by 1
  15. 2025Instance Data Condensation for Image Super-Resolutioncited by 1
  16. 2025A Discrepancy-Based Perspective on Dataset Condensationcited by 1
  17. 2026Condensing Large-Scale Datasets Directly with Minimal Information Losscited by 1
  18. 2023Private Gradient Estimation is Useful for Generative Modelingcited by 0
  19. 2025Beyond Anonymization: Object Scrubbing for Privacy-Preserving 2D and 3D Vision Taskscited by 0
  20. 2026Privacy-Preserving Model Transcription with Differentially Private Synthetic Distillationcited by 0
  21. 2026Multimodal Dataset Distillation Made Simple by Prototype-Guided Data Synthesiscited by 0
  22. 2026DIPBox: A Multi-scale Testing Framework for Tracking Dataset Regenerationcited by 0