mlsanityData to 7 Oct 2026

On the Impossible Safety of Large AI Models

arXiv 2209.15259 ↗First posted 30 Sep 2022cs.LG cs.AI cs.CRcited by 27

Not on the map. Why?

Abstract (arXiv)

Large AI Models (LAIMs), of which large language models are the most prominent recent example, showcase some impressive performance. However they have been empirically found to pose serious security issues. This paper systematizes our knowledge about the fundamental impossibility of building arbitrarily accurate and secure machine learning models. More precisely, we identify key challenging features of many of today's machine learning settings. Namely, high accuracy seems to require memorizing large training datasets, which are often user-generated and highly heterogeneous, with both sensitive information and fake users. We then survey statistical lower bounds that, we argue, constitute a compelling case against the possibility of designing high-accuracy LAIMs with strong security guarantees.

Key facts. Cited by 27 papers in the map (data through 7 Oct 2026). Builds on 81 of its 225 references that are in the map. First posted on arXiv 30 Sep 2022.

Builds on

81 of 225 references.

References · 81

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  4. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,014
  5. 2016Deep Learning with Differential Privacycited by 2,682
  6. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,597
  7. 2020Extracting Training Data from Large Language Modelscited by 2,144
  8. 2019Fine-Tuning Language Models from Human Preferencescited by 2,124
  9. 2019SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systemscited by 1,718
  10. 2016Understanding deep learning requires rethinking generalizationcited by 1,657
  11. 2018Reconciling modern machine learning practice and the bias-variance trade-offcited by 919
  12. 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 847
38 more
  1. 2019Deep Double Descent: Where Bigger Models and More Data Hurtcited by 667
  2. 2020Personalized Federated Learning with Moreau Envelopescited by 612
  3. 2018Federated Learning for Mobile Keyboard Predictioncited by 565
  4. 2017Can Decentralized Algorithms Outperform Centralized Algorithms? A Case Study for Decentralized Parallel Stochastic Gradient Descentcited by 496
  5. 2019Does Learning Require Memorization? A Short Tale about a Long Tailcited by 452
  6. 2019The generalization error of random features regression: Precise asymptotics and double descent curvecited by 410
  7. 2021Large Language Models Can Be Strong Differentially Private Learnerscited by 405
  8. 2021Differentially Private Fine-tuning of Language Modelscited by 392
  9. 2021Persistent Anti-Muslim Bias in Large Language Modelscited by 376
  10. 2021Self-Diagnosis and Self-Debiasing: A Proposal for Reducing Corpus-Based Bias in NLPcited by 336
  11. 2018The Hidden Vulnerability of Distributed Learning in Byzantiumcited by 324
  12. 2013The Composition Theorem for Differential Privacycited by 290
  13. 2020Supervised Contrastive Learning for Pre-trained Language Model Fine-tuningcited by 275
  14. 2020Revisiting Few-sample BERT Fine-tuningcited by 237
  15. 2018Applied Federated Learning: Improving Google Keyboard Query Suggestionscited by 202
  16. 2019Two models of double descent for weak featurescited by 194
  17. 2018To understand deep learning we need to understand kernel learningcited by 186
  18. 2021FedKD: Communication Efficient Federated Learning via Knowledge Distillationcited by 185
  19. 2021Process for Adapting Language Models to Society (PALMS) with Values-Targeted Datasetscited by 179
  20. 2020Recall and Learn: Fine-tuning Deep Pretrained Language Models with Less Forgettingcited by 169
  21. 2020Learning from History for Byzantine Robust Optimizationcited by 137
  22. 2016Agnostic Estimation of Mean and Covariancecited by 131
  23. 2021EF21: A New, Simpler, Theoretically Better, and Practically Faster Error Feedbackcited by 131
  24. 2021When the Curious Abandon Honesty: Federated Learning Is Not Privatecited by 126
  25. 2020Lower Bounds and Optimal Algorithms for Personalized Federated Learningcited by 122
  26. 2019Harmless interpolation of noisy data in regressioncited by 121
  27. 2021Large-Scale Differentially Private BERTcited by 117
  28. 2020Optimal Regularization Can Mitigate Double Descentcited by 107
  29. 2020When Do You Need Billions of Words of Pretraining Data?cited by 104
  30. 2016Minimax Optimal Procedures for Locally Private Estimationcited by 103
  31. 2020When is Memorization of Irrelevant Training Data Necessary for High-Accuracy Learning?cited by 102
  32. 2019Recent Advances in Algorithmic High-Dimensional Robust Statisticscited by 100
  33. 2021An Empirical Survey of the Effectiveness of Debiasing Techniques for Pre-trained Language Modelscited by 96
  34. 2018Does data interpolation contradict statistical optimality?cited by 95
  35. 2013Fingerprinting Codes and the Price of Approximate Differential Privacycited by 81
  36. 2020The Radicalization Risks of GPT-3 and Advanced Neural Language Modelscited by 76
  37. 2020Private Mean Estimation of Heavy-Tailed Distributionscited by 73
  38. 2021Eluding Secure Aggregation in Federated Learning via Model Inconsistencycited by 66

50 most cited of 81 All 81 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 8; 2024: 11; 2025: 6; 2026: 1.

27 papers.

Sort:

By area:

  1. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,565
  2. 2023Large Language Models for Software Engineering: A Systematic Literature Reviewcited by 389
  3. 2023Challenges and Applications of Large Language Modelscited by 251
  4. 2023Can LLM-Generated Misinformation Be Detected?cited by 173
  5. 2024A Comprehensive Study of Knowledge Editing for Large Language Modelscited by 157
  6. 2022Position: Considerations for Differentially Private Learning with Large-Scale Public Pretrainingcited by 86
  7. 2024Large Language Models for Cyber Security: A Systematic Literature Reviewcited by 65
  8. 2023LLM Censorship: A Machine Learning Challenge or a Computer Security Problem?cited by 47
  9. 2023SoK: Memorization in General-Purpose Large Language Modelscited by 36
  10. 2023Citation: A Key to Building Responsible and Accountable Large Language Modelscited by 20
  11. 2025Reality Check: A New Evaluation Ecosystem Is Necessary to Understand AI's Real World Effectscited by 18
  12. 2024Adaptive Gradient Clipping for Robust Federated Learningcited by 10
15 more
  1. 2024Reinforcement Learning from Human Feedback: Whose Culture, Whose Values, Whose Perspectives?cited by 8
  2. 2024On the Workflows and Smells of Leaderboard Operations (LBOps): An Exploratory Study of Foundation Model Leaderboardscited by 6
  3. 2025AI Governance through Marketscited by 5
  4. 2024The poison of dimensionalitycited by 3
  5. 2024A Survey on Offensive AI Within Cybersecuritycited by 3
  6. 2024Inverting Gradient Attacks Makes Powerful Data Poisoningcited by 2
  7. 2025On the Byzantine Fault Tolerance of signSGD with Majority Votecited by 2
  8. 2025What Does 'Human-Centred AI' Mean?cited by 2
  9. 2023Balancing Privacy, Robustness, and Efficiency in Machine Learningcited by 1
  10. 2024Generative AI in Medicinecited by 1
  11. 2024Data Quality in Edge Machine Learning: A State-of-the-Art Surveycited by 0
  12. 2024Building an Ethical and Trustworthy Biomedical AI Ecosystem for the Translational and Clinical Integration of Foundational Modelscited by 0
  13. 2025Approaching the Harm of Gradient Attacks While Only Flipping Labelscited by 0
  14. 2025A Case for Specialisation in Non-Human Entitiescited by 0
  15. 2026Real-World AI Evaluation: How FRAME Generates Systematic Evidence to Resolve the Decision-Maker's Dilemmacited by 0