mlsanityData to 14 Oct 2026

Attacking LLMs and AI Agents: Advertisement Embedding Attacks Against Large Language Models

arXiv 2508.17674 ↗First posted 25 Aug 2025cs.CR cs.AI cs.LGcited by 3

On the map: LLM agents and tool use → Agent threats, guardrails and governance

Abstract (arXiv)

We introduce Advertisement Embedding Attacks (AEA), a new class of LLM security threats that stealthily inject promotional or malicious content into model outputs and AI agents. AEA operate through two low-cost vectors: (1) hijacking third-party service-distribution platforms to prepend adversarial prompts, and (2) publishing back-doored open-source checkpoints fine-tuned with attacker data. Unlike conventional attacks that degrade accuracy, AEA subvert information integrity, causing models to return covert ads, propaganda, or hate speech while appearing normal. We detail the attack pipeline, map five stakeholder victim groups, and present an initial prompt-based self-inspection defense that mitigates these injections without additional model retraining. Our findings reveal an urgent, under-addressed gap in LLM security and call for coordinated detection, auditing, and policy responses from the AI-safety community.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 32 of its 40 references that are in the map. First posted on arXiv 25 Aug 2025.

Builds on

32 of 40 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training10 Jan

Other references · 31

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  3. 2014Explaining and Harnessing Adversarial Examplescited by 7,364
  4. 2013Intriguing properties of neural networkscited by 5,155
  5. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  6. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
  7. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,592
  8. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,281
  9. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,651
  10. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,443
  11. 2023A Survey on Evaluation of Large Language Modelscited by 1,242
  12. 2017BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chaincited by 1,106
19 more
  1. 2017Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoningcited by 1,055
  2. 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 864
  3. 2018How To Backdoor Federated Learningcited by 765
  4. 2016Stealing Machine Learning Models via Prediction APIscited by 619
  5. 2023Jailbreaking ChatGPT via Prompt Engineering: An Empirical Studycited by 498
  6. 2023Scalable Extraction of Training Data from (Production) Language Modelscited by 434
  7. 2020Weight Poisoning Attacks on Pre-trained Modelscited by 341
  8. 2023ChatDoctor: A Medical Chat Model Fine-Tuned on a Large Language Model Meta-AI (LLaMA) Using Medical Domain Knowledgecited by 325
  9. 2021Membership Inference Attacks on Machine Learning: A Surveycited by 321
  10. 2020BadNL: Backdoor Attacks against NLP Models with Semantic-preserving Improvementscited by 211
  11. 2020Adversarial Training for Large Neural Language Modelscited by 111
  12. 2022I Know What You Trained Last Summer: A Survey on Stealing Machine Learning Models and Defencescited by 75
  13. 2018DARTS: Deceiving Autonomous Cars with Toxic Signscited by 54
  14. 2023PoisonPrompt: Backdoor Attack on Prompt-based Large Language Modelscited by 42
  15. 2024SouLLMate: An Adaptive LLM-Driven System for Advanced Mental Health Support and Assessment, Based on a Systematic Application Surveycited by 8
  16. 2024SouLLMate: An Application Enhancing Diverse Mental Health Support with Adaptive LLMs, Prompt Engineering, and RAG Techniquescited by 8
  17. 2024Advancing Mental Health Pre-Screening: A New Custom GPT for Psychological Distress Assessmentcited by 5
  18. 2025A Layered Multi-Expert Framework for Long-Context Mental Health Assessmentscited by 4
  19. 2024LLM Online Spatial-temporal Signal Reconstruction Under Noisecited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026A Survey of Agentic AI and Cybersecurity: Challenges, Opportunities and Use-case Prototypescited by 22
  2. 2026Reference-Based Bias Detection in LLMs via Relative Representations of Hidden Statescited by 0
  3. 2026When Compliance Data Masquerades as Evaluation: Measurement Validity for Deployed AI Systemscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Soft Prompt Threats: Attacking Safety Alignment and Unlearning in Open-Source LLMs through the Embedding Space
  2. 2025Measuring the Security of Mobile LLM Agents under Adversarial Prompts from Untrusted Third-Party Channels
  3. 2024Targeting the Core: A Simple and Effective Method to Attack RAG-based Agents via Direct LLM Manipulation
  4. 2024Breaking Down the Defenses: A Comparative Survey of Attacks on Large Language Models
  5. 2024Coercing LLMs to do and reveal (almost) anything
  6. 2025A Survey of Attacks on Large Language Models
  7. 2025Commercial LLM Agents Are Already Vulnerable to Simple Yet Dangerous Attacks
  8. 2025Breaking to Build: A Threat Model of Prompt-Based Attacks for Securing LLMs