mlsanityData to 14 Oct 2026

10 Security and Privacy Problems in Large Foundation Models

arXiv 2110.15444 ↗First posted 28 Oct 2021cs.CR cs.LGcited by 6

Not on the map. Why?

Abstract (arXiv)

Foundation models--such as GPT, CLIP, and DINO--have achieved revolutionary progress in the past several years and are commonly believed to be a promising approach for general-purpose AI. In particular, self-supervised learning is adopted to pre-train a foundation model using a large amount of unlabeled data. A pre-trained foundation model is like an ``operating system'' of the AI ecosystem. Specifically, a foundation model can be used as a feature extractor for many downstream tasks with little or no labeled training data. Existing studies on foundation models mainly focused on pre-training a better foundation model to improve its performance on downstream tasks in non-adversarial settings, leaving its security and privacy in adversarial settings largely unexplored. A security or privacy issue of a pre-trained foundation model leads to a single point of failure for the AI ecosystem. In this book chapter, we discuss 10 basic security and privacy problems for the pre-trained foundation models, including six confidentiality problems, three integrity problems, and one availability problem. For each problem, we discuss potential opportunities and challenges. We hope our book chapter will inspire future research on the security and privacy of foundation models.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 105 of its 141 references that are in the map. First posted on arXiv 28 Oct 2021.

Builds on

105 of 141 references.

References · 105

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  4. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  5. 2014Explaining and Harnessing Adversarial Examplescited by 7,364
  6. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
  7. 2013Intriguing properties of neural networkscited by 5,155
  8. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
  9. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  10. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,998
  11. 2016Deep Learning with Differential Privacycited by 2,728
  12. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,281
38 more
  1. 2020Extracting Training Data from Large Language Modelscited by 2,189
  2. 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,517
  3. 2019Certified Adversarial Robustness via Randomized Smoothingcited by 1,340
  4. 2020Graph Contrastive Learning with Augmentationscited by 1,208
  5. 2017BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chaincited by 1,106
  6. 2017Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoningcited by 1,055
  7. 2016Practical Black-Box Attacks against Machine Learningcited by 903
  8. 2018How To Backdoor Federated Learningcited by 765
  9. 2016Stealing Machine Learning Models via Prediction APIscited by 619
  10. 2018Fine-Pruning: Defending Against Backdooring Attacks on Deep Neural Networkscited by 580
  11. 2017Provable defenses against adversarial examples via the convex outer adversarial polytopecited by 567
  12. 2018Exploiting Unintended Feature Leakage in Collaborative Learningcited by 520
  13. 2018Poison Frogs! Targeted Clean-Label Poisoning Attacks on Neural Networkscited by 519
  14. 2020Graph Contrastive Learning with Adaptive Augmentationcited by 519
  15. 2018ML-Leaks: Model and Data Independent Membership Inference Attacks and Defenses on Machine Learning Modelscited by 498
  16. 2019Local Model Poisoning Attacks to Byzantine-Robust Federated Learningcited by 493
  17. 2018Spectral Signatures in Backdoor Attackscited by 484
  18. 2018Certified Robustness to Adversarial Examples with Differential Privacycited by 468
  19. 2020GCC: Graph Contrastive Coding for Graph Neural Network Pre-Trainingcited by 463
  20. 2018Adversarial Attacks on Neural Networks for Graph Datacited by 397
  21. 2020Label-Only Membership Inference Attackscited by 388
  22. 2019Hidden Trigger Backdoor Attackscited by 371
  23. 2018Analyzing Federated Learning through an Adversarial Lenscited by 351
  24. 2020Reflection Backdoor: A Natural Backdoor Attack on Deep Neural Networkscited by 349
  25. 2018Certified Defenses against Adversarial Examplescited by 306
  26. 2017Certified Defenses for Data Poisoning Attackscited by 303
  27. 2019Label-Consistent Backdoor Attackscited by 296
  28. 2018Knockoff Nets: Stealing Functionality of Black-Box Modelscited by 292
  29. 2020Systematic Evaluation of Privacy Risks of Machine Learning Modelscited by 292
  30. 2019A backdoor attack against LSTM-based text classification systemscited by 227
  31. 2018Manipulating Machine Learning: Poisoning Attacks and Countermeasures for Regression Learningcited by 225
  32. 2019High Accuracy and High Fidelity Extraction of Neural Networkscited by 214
  33. 2019Detecting AI Trojans Using Meta Neural Analysiscited by 199
  34. 2017Towards Poisoning of Deep Learning Algorithms with Back-gradient Optimizationcited by 197
  35. 2019MemGuard: Defending against Black-Box Membership Inference Attacks via Adversarial Examplescited by 197
  36. 2018Machine Learning with Membership Privacy using Adversarial Regularizationcited by 186
  37. 2020Adversarial Robustness: From Self-Supervised Pre-Training to Fine-Tuningcited by 153
  38. 2020Dynamic Backdoor Attacks Against Machine Learning Modelscited by 151

50 most cited of 105 All 105 (JSON)

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 4; 2025: 1.

6 papers.

  1. 2024A Comprehensive Survey of Foundation Models in Medicinecited by 56
  2. 2023Generative Pre-trained Transformer: A Comprehensive Review on Enabling Technologies, Potential Applications, Emerging Challenges, and Future Directionscited by 55
  3. 2024TUNI: A Textual Unimodal Detector for Identity Inference in CLIP Modelscited by 8
  4. 2024FCert: Certifiably Robust Few-Shot Classification in the Era of Foundation Modelscited by 3
  5. 2025BESA: Boosting Encoder Stealing Attack with Perturbation Recoverycited by 2
  6. 2024Evaluating the Efficacy of Foundational Models: Advancing Benchmarking Practices to Enhance Fine-Tuning Decision-Makingcited by 0