# Ranking Manipulation for Conversational Search Engines

arXiv: https://arxiv.org/abs/2406.03589 · HTML: https://mlsanity.com/paper/2406.03589/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- Placed in Jailbreaking, one of 12 directions in LLM safety and security.
- Builds on 26 of its 75 references that are in the map.
- First posted on arXiv 5 Jun 2024.

## Placed in

- LLM safety and security / [Jailbreaking](https://mlsanity.com/safety/jailbreaking/)

## Builds on (most cited first, 26 of 26)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Universal and Transferable Adversarial Attacks on Aligned Language Models](https://mlsanity.com/paper/2307.15043/) (2023)
- [Retrieval-Augmented Generation for Large Language Models: A Survey](https://mlsanity.com/paper/2312.10997/) (2023)
- [Jailbroken: How Does LLM Safety Training Fail?](https://mlsanity.com/paper/2307.02483/) (2023)
- [Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection](https://mlsanity.com/paper/2302.12173/) (2023)
- [Jailbreaking Black Box Large Language Models in Twenty Queries](https://mlsanity.com/paper/2310.08419/) (2023)
- [Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations](https://mlsanity.com/paper/2312.06674/) (2023)
- [Red Teaming Language Models with Language Models](https://mlsanity.com/paper/2202.03286/) (2022)
- [InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents](https://mlsanity.com/paper/2403.02691/) (2024)
- [Precise Zero-Shot Dense Retrieval without Relevance Labels](https://mlsanity.com/paper/2212.10496/) (2022)
- [Release Strategies and the Social Impacts of Language Models](https://mlsanity.com/paper/1908.09203/) (2019)
- [The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructions](https://mlsanity.com/paper/2404.13208/) (2024)
- [StruQ: Defending Against Prompt Injection with Structured Queries](https://mlsanity.com/paper/2402.06363/) (2024)
- [Is ChatGPT Good at Search? Investigating Large Language Models as Re-Ranking Agents](https://mlsanity.com/paper/2304.09542/) (2023)
- [Hard Prompts Made Easy: Gradient-Based Discrete Optimization for Prompt Tuning and Discovery](https://mlsanity.com/paper/2302.03668/) (2023)
- [FreshLLMs: Refreshing Large Language Models with Search Engine Augmentation](https://mlsanity.com/paper/2310.03214/) (2023)
- [Benchmarking and Defending Against Indirect Prompt Injection Attacks on Large Language Models](https://mlsanity.com/paper/2312.14197/) (2023)
- [ArtPrompt: ASCII Art-based Jailbreak Attacks against Aligned LLMs](https://mlsanity.com/paper/2402.11753/) (2024)
- [Automatically Auditing Large Language Models via Discrete Optimization](https://mlsanity.com/paper/2303.04381/) (2023)
- [Improving Passage Retrieval with Zero-Shot Question Generation](https://mlsanity.com/paper/2204.07496/) (2022)
- [Tensor Trust: Interpretable Prompt Injection Attacks from an Online Game](https://mlsanity.com/paper/2311.01011/) (2023)
- [Jailbreaking GPT-4V via Self-Adversarial Attacks with System Prompts](https://mlsanity.com/paper/2311.09127/) (2023)
- [Hijacking Large Language Models via Adversarial In-Context Learning](https://mlsanity.com/paper/2311.09948/) (2023)
- [Learning a Product Relevance Model from Click-Through Data in E-Commerce](https://mlsanity.com/paper/2102.07098/) (2021)

## Built on by (most cited first, 26 of 26)

- [StealthRank: LLM Ranking Manipulation via Stealthy Prompt Optimization](https://mlsanity.com/paper/2504.05804/) (2025)
- [C-SEO Bench: Does Conversational SEO Work?](https://mlsanity.com/paper/2506.11097/) (2025)
- [Are LLMs Reliable Rankers? Rank Manipulation via Two-Stage Token Optimization](https://mlsanity.com/paper/2510.06732/) (2025)
- [SoK: Privacy Risks and Mitigations in Retrieval-Augmented Generation Systems](https://mlsanity.com/paper/2601.03979/) (2026)
- [Dynamics of Adversarial Attacks on Large Language Model-Based Search Engines](https://mlsanity.com/paper/2501.00745/) (2025)
- [Role-Augmented Intent-Driven Generative Search Engine Optimization](https://mlsanity.com/paper/2508.11158/) (2025)
- [SAGEO Arena: A Realistic Environment for Evaluating Search-Augmented Generative Engine Optimization](https://mlsanity.com/paper/2602.12187/) (2026)
- [Deep-Research Agents Can Be Poisoned via User-Generated Content](https://mlsanity.com/paper/2605.24245/) (2026)
- [CC-GSEO-Bench: A Content-Centric Benchmark for Measuring Source Influence in Generative Search Engines](https://mlsanity.com/paper/2509.05607/) (2025)
- [Controlling Output Rankings in Generative Engines for LLM-based Search](https://mlsanity.com/paper/2602.03608/) (2026)
- [Diagnosing and Repairing Citation Failures in Generative Engine Optimization](https://mlsanity.com/paper/2603.09296/) (2026)
- [SCI-Defense: Defending Manipulation Attacks from Generative Engine Optimization](https://mlsanity.com/paper/2605.21948/) (2026)
- [Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems](https://mlsanity.com/paper/2606.17443/) (2026)
- [A Multi-Agent Perspective on Modern Information Retrieval](https://mlsanity.com/paper/2502.14796/) (2025)
- [Caption Injection for Optimization in Generative Search Engine](https://mlsanity.com/paper/2511.04080/) (2025)
- [How Generative AI Disrupts Search: An Empirical Study of Google Search, Gemini, and AI Overviews](https://mlsanity.com/paper/2604.27790/) (2026)
- [GEO-Bench: Benchmarking Ranking Manipulation in Generative Engine Optimization](https://mlsanity.com/paper/2605.29107/) (2026)
- [When Optimization Becomes Manipulation: Defending Generative Search against Malicious Generative Engine Optimization](https://mlsanity.com/paper/2609.02964/) (2026)
- [The Vulnerability of LLM Rankers to Prompt Injection Attacks](https://mlsanity.com/paper/2602.16752/) (2026)
- [Adversarial Feeds Steer LLM Agent Decisions Against Their Defaults](https://mlsanity.com/paper/2606.00914/) (2026)
- [One Polluted Page Is Enough: Evaluating Web Content Pollution in LLM Recommenders](https://mlsanity.com/paper/2606.13610/) (2026)
- [Breadcrumbing Search Agents](https://mlsanity.com/paper/2608.04565/) (2026)
- [Scoring Without the Engine: Validating a Deterministic, Manipulation-Resistant Content Score for Generative Engines, End to End](https://mlsanity.com/paper/2609.07559/) (2026)
- [CITECHOICE: A Causal Audit of How Document Presentation Redistributes Citation Credit in Agentic Search](https://mlsanity.com/paper/2609.15164/) (2026)
- [Who Is Your Agent Serving? Provider-Side Indirect Prompt Injection in Proactive Agents](https://mlsanity.com/paper/2610.05266/) (2026)
- [What Drives AI Citations? Content, Retrieval and Model Knowledge in Generative Engine Optimization (GEO)](https://mlsanity.com/paper/2610.15033/) (2026)

## Abstract (arXiv)

Major search engine providers are rapidly incorporating Large Language Model (LLM)-generated content in response to user queries. These conversational search engines operate by loading retrieved website text into the LLM context for summarization and interpretation. Recent research demonstrates that LLMs are highly vulnerable to jailbreaking and prompt injection attacks, which disrupt the safety and quality goals of LLMs using adversarial strings. This work investigates the impact of prompt injections on the ranking order of sources referenced by conversational search engines. To this end, we introduce a focused dataset of real-world consumer product websites and formalize conversational search ranking as an adversarial problem. Experimentally, we analyze conversational search rankings in the absence of adversarial injections and show that different LLMs vary significantly in prioritizing product name, document content, and context position. We then present a tree-of-attacks-based jailbreaking technique which reliably promotes low-ranked products. Importantly, these attacks transfer effectively to state-of-the-art conversational search engines such as perplexity.ai. Given the strong financial incentive for website owners to boost their search ranking, we argue that our problem formulation is of critical importance for future robustness work.