# Dynamics of Adversarial Attacks on Large Language Model-Based Search Engines

arXiv: https://arxiv.org/abs/2501.00745 · HTML: https://mlsanity.com/paper/2501.00745/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 38 references that are in the map.
- First posted on arXiv 1 Jan 2025.

## Builds on (most cited first, 20 of 20)

- [Universal and Transferable Adversarial Attacks on Aligned Language Models](https://mlsanity.com/paper/2307.15043/) (2023)
- [Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection](https://mlsanity.com/paper/2302.12173/) (2023)
- [Red Teaming Language Models with Language Models](https://mlsanity.com/paper/2202.03286/) (2022)
- [Universal Adversarial Triggers for Attacking and Analyzing NLP](https://mlsanity.com/paper/1908.07125/) (2019)
- [Tree of Attacks: Jailbreaking Black-Box LLMs Automatically](https://mlsanity.com/paper/2312.02119/) (2023)
- [A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Models](https://mlsanity.com/paper/2405.06211/) (2024)
- [Hard Prompts Made Easy: Gradient-Based Discrete Optimization for Prompt Tuning and Discovery](https://mlsanity.com/paper/2302.03668/) (2023)
- [Benchmarking and Defending Against Indirect Prompt Injection Attacks on Large Language Models](https://mlsanity.com/paper/2312.14197/) (2023)
- [PromptRobust: Towards Evaluating the Robustness of Large Language Models on Adversarial Prompts](https://mlsanity.com/paper/2306.04528/) (2023)
- [Survey of Vulnerabilities in Large Language Models Revealed by Adversarial Attacks](https://mlsanity.com/paper/2310.10844/) (2023)
- [Automatically Auditing Large Language Models via Discrete Optimization](https://mlsanity.com/paper/2303.04381/) (2023)
- [Prompt as Triggers for Backdoor Attack: Examining the Vulnerability in Language Models](https://mlsanity.com/paper/2305.01219/) (2023)
- [GEO: Generative Engine Optimization](https://mlsanity.com/paper/2311.09735/) (2023)
- [When LLM Meets DRL: Advancing Jailbreaking Efficiency via DRL-guided Search](https://mlsanity.com/paper/2406.08705/) (2024)
- [Adversarial Search Engine Optimization for Large Language Models](https://mlsanity.com/paper/2406.18382/) (2024)
- [Manipulating Large Language Models to Increase Product Visibility](https://mlsanity.com/paper/2404.07981/) (2024)
- [Ranking Manipulation for Conversational Search Engines](https://mlsanity.com/paper/2406.03589/) (2024)
- [A Game-Theoretic Taxonomy and Survey of Defensive Deception for Cybersecurity and Privacy](https://mlsanity.com/paper/1712.05441/) (2017)
- [A Game Theoretic Analysis of Additive Adversarial Attacks and Defenses](https://mlsanity.com/paper/2009.06530/) (2020)
- [Algorithmic Transparency with Strategic Users](https://mlsanity.com/paper/2008.09283/) (2020)

## Built on by (most cited first, 5 of 5)

- [StealthRank: LLM Ranking Manipulation via Stealthy Prompt Optimization](https://mlsanity.com/paper/2504.05804/) (2025)
- [Are LLMs Reliable Rankers? Rank Manipulation via Two-Stage Token Optimization](https://mlsanity.com/paper/2510.06732/) (2025)
- [Multimodal Generative Engine Optimization: Rank Manipulation for Vision-Language Model Rankers](https://mlsanity.com/paper/2601.12263/) (2026)
- [When Prompts Become Payloads: A Framework for Mitigating SQL Injection Attacks in Large Language Model-Driven Applications](https://mlsanity.com/paper/2605.10176/) (2026)
- [Agent-Facing Information Design in LLM Tool Registries](https://mlsanity.com/paper/2605.23916/) (2026)

## Abstract (arXiv)

The increasing integration of Large Language Model (LLM) based search engines has transformed the landscape of information retrieval. However, these systems are vulnerable to adversarial attacks, especially ranking manipulation attacks, where attackers craft webpage content to manipulate the LLM's ranking and promote specific content, gaining an unfair advantage over competitors. In this paper, we study the dynamics of ranking manipulation attacks. We frame this problem as an Infinitely Repeated Prisoners' Dilemma, where multiple players strategically decide whether to cooperate or attack. We analyze the conditions under which cooperation can be sustained, identifying key factors such as attack costs, discount rates, attack success rates, and trigger strategies that influence player behavior. We identify tipping points in the system dynamics, demonstrating that cooperation is more likely to be sustained when players are forward-looking. However, from a defense perspective, we find that simply reducing attack success probabilities can, paradoxically, incentivize attacks under certain conditions. Furthermore, defensive measures to cap the upper bound of attack success rates may prove futile in some scenarios. These insights highlight the complexity of securing LLM-based systems. Our work provides a theoretical foundation and practical insights for understanding and mitigating their vulnerabilities, while emphasizing the importance of adaptive security strategies and thoughtful ecosystem design.