Evaluating Deep-Search Agents under Hierarchical Web Evidence Poisoning
On the map: LLM agents and tool use → Agent threats, guardrails and governance
Abstract (arXiv)
Search-augmented LLM agents are increasingly used for consumer decisions, making them vulnerable to Generative Engine Optimization (GEO) poisoning. Existing benchmarks largely measure whether manipulated content is retrieved or endorsed, but do not track whether an agent verifies suspicious evidence, revises adopted claims, or recovers before producing its final recommendation. We introduce HAE-GEO, a benchmark that tracks the full trajectory from exposure to recovery under progressively more persuasive Web poisoning. Agents interact via a multi-turn Search-Scrape interface across three attack levels (L1 direct assertion, L2 contextual camouflage, and L3 apparent corroboration), supported by a controlled corpus of 72,039 clean pages and 770 poisoned pages per level spanning 8 product categories and 154 brands. Evaluation combines deterministic behavioral measures with six semantic rubric dimensions. Evaluating 10 agents, we find three recurring patterns: evidence recognition degrades under the corroboration trap; agentic search improves final resistance without improving evidence recognition or utility; and defense prompting increases verification, yet rarely converts verification into recovery.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 43 of its 65 references that are in the map. First posted on arXiv 5 Sep 2026.
Builds on
43 of 65 references.
Closest ancestors · 5
Same direction, newest first
- 2026FORGE: Research-Trajectory Hijacking Attacks on Deep Research Agents6 Jul
- 2026How Much Can We Trust LLM Search Agents? Measuring Endorsement Vulnerability to Web Content Manipulation15 Jun
- 2026Deep-Research Agents Can Be Poisoned via User-Generated Content22 May
- 2024AgentPoison: Red-teaming LLM Agents via Poisoning Memory or Knowledge Bases17 Jul
- 2024InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents5 Mar
Other references · 38
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
- 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
- 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,276
- 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 2,006
- 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,986
- 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,947
- 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,443
- 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,332
- 2026DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligencecited by 1,040
- 2024From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judgecited by 494
- 2024WorkArena: How Capable Are Web Agents at Solving Common Knowledge Work Tasks?cited by 363
26 more
- 2024AgentBoard: An Analytical Evaluation Board of Multi-turn LLM Agentscited by 339
- 2024PoisonedRAG: Knowledge Corruption Attacks to Retrieval-Augmented Generation of Large Language Modelscited by 267
- 2025DeepResearch Bench: A Comprehensive Benchmark for Deep Research Agentscited by 248
- 2024Judging the Judges: Evaluating Alignment and Vulnerabilities in LLMs-as-Judgescited by 205
- 2023Poisoning Retrieval Corpora by Injecting Adversarial Passagescited by 159
- 2024An Empirical Study of LLM-as-a-Judge for LLM Evaluation: Fine-tuned Judge Model is not a General Substitute for GPT-4cited by 94
- 2023GEO: Generative Engine Optimizationcited by 56
- 2024Adversarial Search Engine Optimization for Large Language Modelscited by 40
- 2024JuStRank: Benchmarking LLM Judges for System Rankingcited by 29
- 2025Search-Time Data Contaminationcited by 19
- 2025Benchmarking Poisoning Attacks against Retrieval-Augmented Generationcited by 18
- 2024Adversarial Decoding: Generating Readable Documents for Adversarial Objectivescited by 16
- 2025C-SEO Bench: Does Conversational SEO Work?cited by 11
- 2024ProductAgent: Benchmarking Conversational Product Search Agent with Asking Clarification Questionscited by 10
- 2026ShopSimulator: Evaluating and Exploring RL-Driven LLM Agent for Shopping Assistantscited by 10
- 2026DeepFact: Co-Evolving Benchmarks and Agents for Deep Research Factualitycited by 10
- 2025ShoppingComp: Are LLMs Really Ready for Your Shopping Cart?cited by 9
- 2026Search-Time Contamination in Deep Research Agents: Measuring Performance Inflation in Public Benchmark Evaluationcited by 9
- 2026DREAM: Deep Research Evaluation with Agentic Metricscited by 7
- 2026DR-Arena: an Automated Evaluation Framework for Deep Research Agentscited by 6
- 2025MIRAGE: Misleading Retrieval-Augmented Generation via Black-box and Query-agnostic Poisoning Attackscited by 4
- 2026SafeGEO: Understanding Generative Engine Optimization Risks in Recommendation Agentscited by 4
- 2026Is Deep Research Reliable? Misleading Knowledge Induces False Conclusionscited by 4
- 2026EcoGEO: Trajectory-Aware Evidence Ecosystems for Web-Enabled LLM Search Agentscited by 3
- 2026SCI-Defense: Defending Manipulation Attacks from Generative Engine Optimizationcited by 3
- 2026GEO-Bench: Benchmarking Ranking Manipulation in Generative Engine Optimizationcited by 2
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026Do Web Agents Investigate Before They Decide?
- 2026EviGraph: Towards Verifiable Evidence Construction for Information-Seeking Agents
- 2026MERRIN: A Benchmark for Multimodal Evidence Retrieval and Reasoning in Noisy Web Environments
- 2026The Synthetic Web: Adversarially-Curated Mini-Internets for Diagnosing Epistemic Weaknesses of Language Agents
- 2025It's a TRAP! Task-Redirecting Agent Persuasion Benchmark for Web Agents
- 2025Mind2Web 2: Evaluating Agentic Search with Agent-as-a-Judge
- 2026Relevance as a Vulnerability: How Web Retrieval Degrades Safety Alignment in LLM Agents
- 2026RiskChainBench: A Benchmark for Obfuscated Platform Message Restoration and Evidence-Grounded Web Investigation