mlsanityData to 7 Oct 2026

Machine Learning Models that Remember Too Much

arXiv 1709.07886 ↗First posted 22 Sep 2017cs.CR cs.LGcited by 165

Not on the map. Why?

Cited by 37 in LLM safety and security, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Machine learning (ML) is becoming a commodity. Numerous ML frameworks and services are available to data holders who are not ML experts but want to train predictive models on their data. It is important that ML models trained on sensitive inputs (e.g., personal images or documents) not leak too much information about the training data. We consider a malicious ML provider who supplies model-training code to the data holder, does not observe the training, but then obtains white- or black-box access to the resulting model. In this setting, we design and implement practical algorithms, some of them very similar to standard ML techniques such as regularization and data augmentation, that "memorize" information about the training dataset in the model yet the model is as accurate and predictive as a conventionally trained model. We then explain how the adversary can extract memorized information from the model. We evaluate our techniques on standard ML tasks for image classification (CIFAR10), face recognition (LFW and FaceScrub), and text analysis (20 Newsgroups and IMDB). In all cases, we show how our algorithms create models that have high predictive power yet allow accurate extraction of subsets of their training data.

Key facts. Cited by 165 papers in the map (data through 7 Oct 2026). First posted on arXiv 22 Sep 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 32; 2022: 36; 2023: 28; 2024: 35; 2025: 21; 2026: 13.

165 papers.

Sort:

By area:

  1. 2021Membership Inference Attacks on Machine Learning: A Surveycited by 306
  2. 2021Enhanced Membership Inference Attacks against Machine Learning Modelscited by 237
  3. 2021Adversary Instantiation: Lower Bounds for Differentially Private Machine Learningcited by 193
  4. 2022Truth Serum: Poisoning Machine Learning Models to Reveal Their Secretscited by 94
  5. 2023Sentence Embedding Leaks More Information than You Expect: Generative Embedding Inversion Attack to Recover the Whole Sentencecited by 81
  6. 2021Mitigating Membership Inference Attacks by Self-Distillation Through a Novel Ensemble Architecturecited by 63
  7. 2021Non-Transferable Learning: A New Approach for Model Ownership Verification and Applicability Authorizationcited by 50
  8. 2021Survey: Leakage and Privacy at Inference Timecited by 44
  9. 2021Gradient-Leakage Resilient Federated Learningcited by 40
  10. 2022Unbiased Supervised Contrastive Learningcited by 40
  11. 2021Leave-one-out Unfairnesscited by 36
  12. 2021Quantifying and Mitigating Privacy Risks of Contrastive Learningcited by 35
38 more
  1. 2022RelaxLoss: Defending Membership Inference Attacks without Losing Utilitycited by 35
  2. 2021Source Inference Attacks in Federated Learningcited by 31
  3. 2024Membership Inference Attacks against Large Vision-Language Modelscited by 31
  4. 2024Privacy Leakage on DNNs: A Survey of Model Inversion Attacks and Defensescited by 30
  5. 2024A Review on Machine Unlearningcited by 30
  6. 2022Auditing Membership Leakages of Multi-Exit Networkscited by 26
  7. 2023Privacy and Fairness in Federated Learning: on the Perspective of Trade-offcited by 26
  8. 2021This Person (Probably) Exists. Identity Membership Attacks Against GAN Generated Facescited by 25
  9. 2021The Role of Adaptive Optimizers for Honest Private Hyperparameter Selectioncited by 25
  10. 2021CAPE: Context-Aware Private Embeddings for Private Language Learningcited by 22
  11. 2024Few-shot Adaptation of Medical Vision-Language Modelscited by 22
  12. 2023Backdoor Attacks Against Dataset Distillationcited by 20
  13. 2024Privacy Backdoors: Stealing Data with Corrupted Pretrained Modelscited by 20
  14. 2022Deletion Inference, Reconstruction, and Compliance in Machine (Un)Learningcited by 19
  15. 2021Honest-but-Curious Nets: Sensitive Attributes of Private Inputs Can Be Secretly Coded into the Classifiers' Outputscited by 18
  16. 2021Reversible Watermarking in Deep Convolutional Neural Networks for Integrity Authenticationcited by 17
  17. 2021Stealthy Backdoors as Compression Artifactscited by 17
  18. 2021Gradient Leakage Attack Resilient Deep Learningcited by 17
  19. 2023Towards Foundation Models and Few-Shot Parameter-Efficient Fine-Tuning for Volumetric Organ Segmentationcited by 17
  20. 2024A Closer Look at GAN Priors: Exploiting Intermediate Features for Enhanced Model Inversion Attackscited by 17
  21. 2025Vid-SME: Membership Inference Attacks against Large Video Understanding Modelscited by 17
  22. 2022Analyzing Privacy Leakage in Machine Learning via Multiple Hypothesis Testing: A Lesson From Fanocited by 16
  23. 2021Generative Adversarial Networks: A Survey Towards Private and Secure Applicationscited by 15
  24. 2021Teacher Model Fingerprinting Attacks Against Transfer Learningcited by 15
  25. 2022Federated Multi-organ Segmentation with Inconsistent Labelscited by 15
  26. 2024A General Framework for Data-Use Auditing of ML Modelscited by 15
  27. 2022On the Privacy Effect of Data Enhancement via the Lens of Memorizationcited by 14
  28. 2023Model Barrier: A Compact Un-Transferable Isolation Domain for Model Intellectual Property Protectioncited by 14
  29. 2023Unstoppable Attack: Label-Only Model Inversion via Conditional Diffusion Modelcited by 14
  30. 2023The Inadequacy of Similarity-based Privacy Metrics: Privacy Attacks against "Truly Anonymous" Synthetic Datasetscited by 14
  31. 2021Differentially Private Imaging via Latent Space Manipulationcited by 13
  32. 2021Privacy and Trust Redefined in Federated Machine Learningcited by 13
  33. 2022Federated Learning and Meta Learning: Approaches, Applications, and Directionscited by 12
  34. 2022Adap DP-FL: Differentially Private Federated Learning with Adaptive Noisecited by 12
  35. 2024Trustworthy Distributed AI Systems: Robustness, Privacy, and Governancecited by 12
  36. 2021Bounding Information Leakage in Machine Learningcited by 11
  37. 2022You Are What You Write: Preserving Privacy in the Era of Large Language Modelscited by 11
  38. 2022Data Isotopes for Data Provenance in DNNscited by 11

50 most cited of 165 All 165 (JSON)