mlsanityData to 14 Oct 2026

Multi-Agent Framework for Threat Mitigation and Resilience in AI-Based Systems

arXiv 2512.23132 ↗First posted 29 Dec 2025cs.CR cs.LG cs.MAcited by 2

On the map: LLM agents and tool use → Agent threats, guardrails and governance

also in 1 more
Abstract (arXiv)

Machine learning (ML) underpins foundation models in finance, healthcare, and critical infrastructure, making them targets for data poisoning, model extraction, prompt injection, automated jailbreaking, and preference-guided black-box attacks that exploit model comparisons. Larger models can be more vulnerable to introspection-driven jailbreaks and cross-modal manipulation. Traditional cybersecurity lacks ML-specific threat modeling for foundation, multimodal, and RAG systems. Objective: Characterize ML security risks by identifying dominant TTPs, vulnerabilities, and targeted lifecycle stages. Methods: We extract 93 threats from MITRE ATLAS (26), AI Incident Database (12), and literature (55), and analyze 854 GitHub/Python repositories. A multi-agent RAG system (ChatGPT-4o, temp 0.4) mines 300+ articles to build an ontology-driven threat graph linking TTPs, vulnerabilities, and stages. Results: We identify unreported threats including commercial LLM API model stealing, parameter memorization leakage, and preference-guided text-only jailbreaks. Dominant TTPs include MASTERKEY-style jailbreaking, federated poisoning, diffusion backdoors, and preference optimization leakage, mainly impacting pre-training and inference. Graph analysis reveals dense vulnerability clusters in libraries with poor patch propagation. Conclusion: Adaptive, ML-specific security frameworks, combining dependency hygiene, threat intelligence, and monitoring, are essential to mitigate supply-chain and inference risks across the ML lifecycle.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 47 of its 226 references that are in the map. First posted on arXiv 29 Dec 2025.

Builds on

47 of 226 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025Design Patterns for Securing LLM Agents against Prompt Injections10 Jun
  2. 2024The Dark Side of Function Calling: Pathways to Jailbreaking Large Language Models25 Jul

Other references · 45

  1. 2017BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chaincited by 1,106
  2. 2019GNNExplainer: Generating Explanations for Graph Neural Networkscited by 737
  3. 2023Baseline Defenses for Adversarial Attacks Against Aligned Language Modelscited by 651
  4. 2023Prompt Injection attack against LLM-integrated Applicationscited by 605
  5. 2017Feature Squeezing: Detecting Adversarial Examples in Deep Neural Networkscited by 486
  6. 2022Diffusion Models for Adversarial Purificationcited by 476
  7. 2019On Evaluating Adversarial Robustnesscited by 447
  8. 2016Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samplescited by 446
  9. 2023Visual Adversarial Examples Jailbreak Aligned Large Language Modelscited by 328
  10. 2023MasterKey: Automated Jailbreak Across Multiple Large Language Model Chatbotscited by 183
  11. 2024Stealing Part of a Production Language Modelcited by 159
  12. 2020Backdoor Attacks and Countermeasures on Deep Learning: A Comprehensive Reviewcited by 123
33 more
  1. 2023Explore, Establish, Exploit: Red Teaming Language Models from Scratchcited by 114
  2. 2020Sponge Examples: Energy-Latency Attacks on Neural Networkscited by 111
  3. 2018Adversarial Attacks Against Automatic Speech Recognition Systems via Psychoacoustic Hidingcited by 85
  4. 2023FLIRT: Feedback Loop In-context Red Teamingcited by 83
  5. 2020Imitation Attacks and Defenses for Black-box Machine Translation Systemscited by 81
  6. 2024Best-of-N Jailbreakingcited by 76
  7. 2024Here Comes The AI Worm: Unleashing Zero-click Worms that Target GenAI-Powered Applicationscited by 59
  8. 2023TrustGPT: A Benchmark for Trustworthy and Responsible Large Language Modelscited by 58
  9. 2020TextAttack: A Framework for Adversarial Attacks, Data Augmentation, and Adversarial Training in NLPcited by 45
  10. 2021Data Poisoning Attacks to Deep Learning Based Recommender Systemscited by 43
  11. 2019Practical Hidden Voice Attacks against Speech and Speaker Recognition Systemscited by 36
  12. 2025The Traitors: Deception and Trust in Multi-Agent Language Model Simulationscited by 34
  13. 2024PoisonBench: Assessing Large Language Model Vulnerability to Data Poisoningcited by 33
  14. 2025The Jailbreak Tax: How Useful are Your Jailbreak Outputs?cited by 28
  15. 2025Leaky Thoughts: Large Reasoning Models Are Not Private Thinkerscited by 28
  16. 2025LLMs are Vulnerable to Malicious Prompts Disguised as Scientific Languagecited by 17
  17. 2019Detecting Adversarial Examples and Other Misclassifications in Neural Networks by Introspectioncited by 16
  18. 2025Position: Adversarial ML for LLMs Is Not Making Any Progresscited by 15
  19. 2024Gradient-based Jailbreak Images for Multimodal Fusion Modelscited by 14
  20. 2023Safeguarding Crowdsourcing Surveys from ChatGPT with Prompt Injectioncited by 11
  21. 2024Prompt Obfuscation for Large Language Modelscited by 10
  22. 2024From Vulnerabilities to Remediation: A Systematic Literature Review of LLMs in Code Securitycited by 10
  23. 2025LLM4CVE: Enabling Iterative Automated Vulnerability Repair with Large Language Modelscited by 9
  24. 2025Probing Latent Subspaces in LLM for AI Security: Identifying and Manipulating Adversarial Statescited by 9
  25. 2023Vertical Federated Learning: Taxonomies, Threats, and Prospectscited by 8
  26. 2023Chatbots to ChatGPT in a Cybersecurity Space: Evolution, Vulnerabilities, Attacks, Challenges, and Future Recommendationscited by 8
  27. 2023Deconstructing Classifiers: Towards A Data Reconstruction Attack Against Text Classification Modelscited by 6
  28. 2023Knowledge Graphs Queryingcited by 5
  29. 2023Protect Your Prompts: Protocols for IP Protection in LLM Applicationscited by 4
  30. 2024Foundation Models in Federated Learning: Assessing Backdoor Vulnerabilitiescited by 3
  31. 2025VulRG: Multi-Level Explainable Vulnerability Patch Ranking for Complex Systems Using Graphscited by 3
  32. 2025From Macro to Micro: Probing Dataset Diversity in Language Model Fine-Tuningcited by 3
  33. 2023Cybersecurity Threat Hunting and Vulnerability Analysis Using a Neo4j Graph Database of Open Source Intelligencecited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026From Spark to Fire: Modeling and Mitigating Error Cascades in LLM-Based Multi-Agent Collaborationcited by 14
  2. 2026Signals in the Noise: Open Source Intelligence (OSINT) for AI Loss of Control Detectioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026AI Security in the Foundation Model Era: A Comprehensive Survey from a Unified Perspective
  2. 2026Security Considerations for Multi-agent Systems
  3. 2026SoK: The Attack Surface of Agentic AI - Tools and Autonomy
  4. 2025Breaking Agent Backbones: Evaluating the Security of Backbone LLMs in AI Agents
  5. 2025ATAG: AI-Agent Application Threat Assessment with Attack Graphs
  6. 2025Bridging AI and Software Security: A Comparative Vulnerability Assessment of LLM Agent Deployment Paradigms
  7. 2025MALCDF: A Distributed Multi-Agent LLM Framework for Real-Time Cyber
  8. 2025Exposing Weak Links in Multi-Agent Systems under Adversarial Prompting