mlsanityData to 14 Oct 2026

Ranking Manipulation for Conversational Search Engines

arXiv 2406.03589 ↗First posted 5 Jun 2024cs.CLcited by 26

On the map: LLM safety and security → Jailbreaking

Abstract (arXiv)

Major search engine providers are rapidly incorporating Large Language Model (LLM)-generated content in response to user queries. These conversational search engines operate by loading retrieved website text into the LLM context for summarization and interpretation. Recent research demonstrates that LLMs are highly vulnerable to jailbreaking and prompt injection attacks, which disrupt the safety and quality goals of LLMs using adversarial strings. This work investigates the impact of prompt injections on the ranking order of sources referenced by conversational search engines. To this end, we introduce a focused dataset of real-world consumer product websites and formalize conversational search ranking as an adversarial problem. Experimentally, we analyze conversational search rankings in the absence of adversarial injections and show that different LLMs vary significantly in prioritizing product name, document content, and context position. We then present a tree-of-attacks-based jailbreaking technique which reliably promotes low-ranked products. Importantly, these attacks transfer effectively to state-of-the-art conversational search engines such as perplexity.ai. Given the strong financial incentive for website owners to boost their search ranking, we argue that our problem formulation is of critical importance for future robustness work.

Key facts. Cited by 26 papers in the map (data through 14 Oct 2026). Placed in Jailbreaking, one of 12 directions in LLM safety and security. Builds on 26 of its 75 references that are in the map. First posted on arXiv 5 Jun 2024.

Builds on

26 of 75 references.

Closest ancestors · 4

Same direction, newest first

  1. 2023Jailbreaking GPT-4V via Self-Adversarial Attacks with System Prompts15 Nov
  2. 2023Jailbreaking Black Box Large Language Models in Twenty Queries12 Oct
  3. 2023Universal and Transferable Adversarial Attacks on Aligned Language Models27 Jul
  4. 2023Jailbroken: How Does LLM Safety Training Fail?5 Jul

Other references · 22

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  3. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  4. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,212
  5. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,443
  6. 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,207
  7. 2022Red Teaming Language Models with Language Modelscited by 1,026
  8. 2024InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agentscited by 538
  9. 2022Precise Zero-Shot Dense Retrieval without Relevance Labelscited by 517
  10. 2019Release Strategies and the Social Impacts of Language Modelscited by 513
  11. 2024The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructionscited by 493
  12. 2024StruQ: Defending Against Prompt Injection with Structured Queriescited by 375
10 more
  1. 2023Is ChatGPT Good at Search? Investigating Large Language Models as Re-Ranking Agentscited by 360
  2. 2023Hard Prompts Made Easy: Gradient-Based Discrete Optimization for Prompt Tuning and Discoverycited by 336
  3. 2023FreshLLMs: Refreshing Large Language Models with Search Engine Augmentationcited by 318
  4. 2023Benchmarking and Defending Against Indirect Prompt Injection Attacks on Large Language Modelscited by 264
  5. 2024ArtPrompt: ASCII Art-based Jailbreak Attacks against Aligned LLMscited by 216
  6. 2023Automatically Auditing Large Language Models via Discrete Optimizationcited by 191
  7. 2022Improving Passage Retrieval with Zero-Shot Question Generationcited by 165
  8. 2023Tensor Trust: Interpretable Prompt Injection Attacks from an Online Gamecited by 145
  9. 2023Hijacking Large Language Models via Adversarial In-Context Learningcited by 45
  10. 2021Learning a Product Relevance Model from Click-Through Data in E-Commercecited by 22

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 8; 2026: 18.

26 papers.

Sort:

By area:

  1. 2025StealthRank: LLM Ranking Manipulation via Stealthy Prompt Optimizationcited by 12
  2. 2025C-SEO Bench: Does Conversational SEO Work?cited by 11
  3. 2025Are LLMs Reliable Rankers? Rank Manipulation via Two-Stage Token Optimizationcited by 9
  4. 2026SoK: Privacy Risks and Mitigations in Retrieval-Augmented Generation Systemscited by 7
  5. 2025Dynamics of Adversarial Attacks on Large Language Model-Based Search Enginescited by 5
  6. 2025Role-Augmented Intent-Driven Generative Search Engine Optimizationcited by 5
  7. 2026SAGEO Arena: A Realistic Environment for Evaluating Search-Augmented Generative Engine Optimizationcited by 5
  8. 2026Deep-Research Agents Can Be Poisoned via User-Generated Contentcited by 4
  9. 2025CC-GSEO-Bench: A Content-Centric Benchmark for Measuring Source Influence in Generative Search Enginescited by 3
  10. 2026Controlling Output Rankings in Generative Engines for LLM-based Searchcited by 3
  11. 2026Diagnosing and Repairing Citation Failures in Generative Engine Optimizationcited by 3
  12. 2026SCI-Defense: Defending Manipulation Attacks from Generative Engine Optimizationcited by 3
14 more
  1. 2026Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systemscited by 3
  2. 2025A Multi-Agent Perspective on Modern Information Retrievalcited by 2
  3. 2025Caption Injection for Optimization in Generative Search Enginecited by 2
  4. 2026How Generative AI Disrupts Search: An Empirical Study of Google Search, Gemini, and AI Overviewscited by 2
  5. 2026GEO-Bench: Benchmarking Ranking Manipulation in Generative Engine Optimizationcited by 2
  6. 2026When Optimization Becomes Manipulation: Defending Generative Search against Malicious Generative Engine Optimizationcited by 1
  7. 2026The Vulnerability of LLM Rankers to Prompt Injection Attackscited by 0
  8. 2026Adversarial Feeds Steer LLM Agent Decisions Against Their Defaultscited by 0
  9. 2026One Polluted Page Is Enough: Evaluating Web Content Pollution in LLM Recommenderscited by 0
  10. 2026Breadcrumbing Search Agentscited by 0
  11. 2026Scoring Without the Engine: Validating a Deterministic, Manipulation-Resistant Content Score for Generative Engines, End to Endcited by 0
  12. 2026CITECHOICE: A Causal Audit of How Document Presentation Redistributes Citation Credit in Agentic Searchcited by 0
  13. 2026Who Is Your Agent Serving? Provider-Side Indirect Prompt Injection in Proactive Agentscited by 0
  14. 2026What Drives AI Citations? Content, Retrieval and Model Knowledge in Generative Engine Optimization (GEO)cited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Led to Mislead: Adversarial Content Injection for Attacks on Neural Ranking Models
  2. 2024Evaluating Robustness of Generative Search Engine on Adversarial Factual Questions
  3. 2025Topic-FlipRAG: Topic-Orientated Adversarial Opinion Manipulation Attacks to Retrieval-Augmented Generation Models
  4. 2025Adversarial Attacks against Neural Ranking Models via In-Context Learning
  5. 2024Attack-in-the-Chain: Bootstrapping Large Language Models for Attacks Against Black-box Neural Ranking Models
  6. 2026Aligning Large Language Models with Searcher Preferences
  7. 2025RobustMask: Certified Robustness against Adversarial Neural Ranking Attack via Randomized Masking
  8. 2025Refining Input Guardrails: Enhancing LLM-as-a-Judge Efficiency Through Chain-of-Thought Fine-Tuning and Alignment