mlsanityData to 7 Oct 2026

AI Security in the Foundation Model Era: A Comprehensive Survey from a Unified Perspective

arXiv 2603.24857 ↗First posted 25 Mar 2026cs.CR cs.AI cs.CLcited by 1

On the map: LLM safety and security → Backdoor and data poisoning attacks

Abstract (arXiv)

As machine learning (ML) systems expand in both scale and functionality, the security landscape has become increasingly complex, with a proliferation of attacks and defenses. However, existing studies largely treat these threats in isolation, lacking a coherent framework to expose their shared principles and interdependencies. This fragmented view hinders systematic understanding and limits the design of comprehensive defenses. Crucially, the two foundational assets of ML -- data and models -- are no longer independent; vulnerabilities in one directly compromise the other. The absence of a holistic framework leaves open questions about how these bidirectional risks propagate across the ML pipeline. To address this critical gap, we propose a unified closed-loop threat taxonomy that explicitly frames model-data interactions along four directional axes. Our framework offers a principled lens for analyzing and defending foundation models. The resulting four classes of security threats represent distinct but interrelated categories of attacks: (1) DatarightarrowData (DrightarrowD): including data decryption attacks and watermark removal attacks; (2) DatarightarrowModel (DrightarrowM): including poisoning, harmful fine-tuning attacks, and jailbreak attacks; (3) ModelrightarrowData (MrightarrowD): including model inversion, membership inference attacks, and training data extraction attacks; (4) ModelrightarrowModel (MrightarrowM): including model extraction attacks. Our unified framework elucidates the underlying connections among these security threats and establishes a foundation for developing scalable, transferable, and cross-modal security strategies, particularly within the landscape of foundation models.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Backdoor and data poisoning attacks, one of 12 directions in LLM safety and security. Builds on 203 of its 301 references that are in the map. First posted on arXiv 25 Mar 2026.

Builds on

203 of 301 references.

Closest ancestors · 5

Same direction, newest first

  1. 2024Scaling Trends for Data Poisoning in LLMs6 Aug
  2. 2022Amplifying Membership Exposure via Data Poisoning1 Nov
  3. 2022Wild Patterns Reloaded: A Survey of Machine Learning Security against Training Data Poisoning4 May
  4. 2022Poisoning Attacks and Defenses on Artificial Intelligence: A Survey21 Feb
  5. 2021De-Pois: An Attack-Agnostic Defense against Data Poisoning Attacks8 May

Other references · 198

  1. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  4. 2016Deep Learning with Differential Privacycited by 2,682
  5. 2020Extracting Training Data from Large Language Modelscited by 2,144
  6. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,259
  7. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,106
  8. 2018Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labelscited by 1,053
  9. 2017BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chaincited by 1,052
  10. 2017Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoningcited by 1,012
  11. 2018Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labelscited by 994
  12. 2019When Does Label Smoothing Help?cited by 985
38 more
  1. 2018Variational image compression with a scale hyperpriorcited by 910
  2. 2016Practical Black-Box Attacks against Machine Learningcited by 893
  3. 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 847
  4. 2022Quantifying Memorization Across Neural Language Modelscited by 755
  5. 2023Extracting Training Data from Diffusion Modelscited by 723
  6. 2021MedMNIST v2 -- A large-scale lightweight benchmark for 2D and 3D biomedical image classificationcited by 682
  7. 2018Learning to Reweight Examples for Robust Deep Learningcited by 670
  8. 2023A Watermark for Large Language Modelscited by 667
  9. 2017MentorNet: Learning Data-Driven Curriculum for Very Deep Neural Networks on Corrupted Labelscited by 652
  10. 2016Stealing Machine Learning Models via Prediction APIscited by 602
  11. 2020Learned Image Compression with Discretized Gaussian Mixture Likelihoods and Attention Modulescited by 570
  12. 2018Poison Frogs! Targeted Clean-Label Poisoning Attacks on Neural Networkscited by 500
  13. 2024JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Modelscited by 494
  14. 2019STRIP: A Defence Against Trojan Attacks on Deep Neural Networkscited by 437
  15. 2014Understanding Deep Image Representations by Inverting Themcited by 399
  16. 2023Safety-Tuned LLaMAs: Lessons From Improving the Safety of Large Language Models that Follow Instructionscited by 377
  17. 2023Detecting Pretraining Data from Large Language Modelscited by 375
  18. 2018Turning Your Weakness Into a Strength: Watermarking Deep Neural Networks by Backdooringcited by 334
  19. 2017Deep Models Under the GAN: Information Leakage from Collaborative Deep Learningcited by 326
  20. 2020Weight Poisoning Attacks on Pre-trained Modelscited by 318
  21. 2021Neural Attention Distillation: Erasing Backdoor Triggers from Deep Neural Networkscited by 313
  22. 2021Membership Inference Attacks on Machine Learning: A Surveycited by 306
  23. 2018Knockoff Nets: Stealing Functionality of Black-Box Modelscited by 285
  24. 2023Poisoning Web-Scale Training Datasets is Practicalcited by 276
  25. 2021Anti-Backdoor Learning: Training Clean Models on Poisoned Datacited by 258
  26. 2018Generating Adversarial Examples with Adversarial Networkscited by 256
  27. 2023Membership Inference Attacks against Language Models via Neighbourhood Comparisoncited by 249
  28. 2019Adversarial Attacks and Defenses in Images, Graphs and Text: A Reviewcited by 234
  29. 2024Jailbreak Attacks and Defenses Against Large Language Models: A Surveycited by 215
  30. 2019High Accuracy and High Fidelity Extraction of Neural Networkscited by 208
  31. 2020MedMNIST Classification Decathlon: A Lightweight AutoML Benchmark for Medical Image Analysiscited by 206
  32. 2017Attentive Generative Adversarial Network for Raindrop Removal from a Single Imagecited by 204
  33. 2018Spatially Transformed Adversarial Examplescited by 195
  34. 2018Machine Learning with Membership Privacy using Adversarial Regularizationcited by 177
  35. 2023How Robust is Google's Bard to Adversarial Image Attacks?cited by 177
  36. 2015Inverting Visual Representations with Convolutional Networkscited by 168
  37. 2024Safety Fine-Tuning at (Almost) No Cost: A Baseline for Vision Large Language Modelscited by 160
  38. 2018PRADA: Protecting against DNN Model Stealing Attackscited by 159

50 most cited of 198 All 198 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Towards a Data-Parameter Correspondence for LLMs: A Preliminary Discussioncited by 0