Do Web Agents Investigate Before They Decide?
On the map: LLM agents and tool use → Web, search, and research agents
Abstract (arXiv)
Autonomous web agents are increasingly deployed in moderation and policy enforcement, where correct decisions often depend on evidence that is not immediately visible and must be actively investigated. Yet existing benchmarks largely assume task critical information is immediately accessible. They do not measure investigative competence: recognizing when visible context is insufficient, retrieving hidden evidence, and integrating it into a final decision. We introduce MIRAGE, a benchmark of 750 multi step decision tasks across three domains: Wikipedia Forensics, Shopping Admin adjudication, and Reddit Moderation. Each task has two layers: a visible surface context that often points to the wrong action, and a hidden context, reachable only by active investigation, that contains the decisive evidence. We decompose agent performance into Investigation, Reasoning, and Decision Accuracy, complemented by an Investigative Hallucination Rate. We evaluate eight LLM agents across two model generations. Three patterns emerge. First, agents reach relevant pages but rarely extract the decisive evidence on them. Second, procedural hints improve investigation but do not consistently improve decisions on Wikipedia tasks, where decisive evidence often contradicts surface impressions. Third, 12.6% of trajectories cite fabricated facts. We call these the Navigation Discovery Gap, Collapse under Contradiction, and Investigative Hallucination. These patterns persist across model scale, generation, and reasoning architecture.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use. Builds on 26 of its 29 references that are in the map. First posted on arXiv 5 Feb 2026.
Builds on
26 of 29 references.
Closest ancestors · 5
Same direction, newest first
- 2025WebGraphEval: Multi-Turn Trajectory Evaluation for Web Agents using Graph Representation22 Oct
- 2025Do LLM Agents Know How to Ground, Recover, and Assess? A Benchmark for Epistemic Competence in Information-Seeking Agents26 Sep
- 2024WebCanvas: Benchmarking Web Agents in Online Environments18 Jun
- 2023Mind2Web: Towards a Generalist Agent for the Web9 Jun
- 2022WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agents4 Jul
Other references · 21
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
- 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
- 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
- 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
- 2023A Survey on Large Language Model based Autonomous Agentscited by 2,203
- 2022Survey of Hallucination in Natural Language Generationcited by 2,056
- 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,422
- 2023GAIA: a benchmark for General AI Assistantscited by 1,084
- 2023Enabling Large Language Models to Generate Text with Citationscited by 584
- 2020The Hateful Memes Challenge: Detecting Hate Speech in Multimodal Memescited by 485
9 more
- 2022LLM-Planner: Few-Shot Grounded Planning for Embodied Agents with Large Language Modelscited by 459
- 2024AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agentscited by 406
- 2023The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A"cited by 399
- 2025The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexitycited by 298
- 2025Plan-and-Act: Improving Planning of Agents for Long-Horizon Taskscited by 190
- 2023Do the Rewards Justify the Means? Measuring Trade-Offs Between Rewards and Ethical Behavior in the MACHIAVELLI Benchmarkcited by 174
- 2025SafeArena: Evaluating the Safety of Autonomous Web Agentscited by 97
- 2025CARES: Comprehensive Evaluation of Safety and Adversarial Robustness in Medical LLMscited by 26
- 2025Robust Hallucination Detection in LLMs via Adaptive Token Selectioncited by 19
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Evaluating Deep-Search Agents under Hierarchical Web Evidence Poisoning
- 2025REAL: Benchmarking Autonomous Agents on Deterministic Simulations of Real Websites
- 2026When Web Agents Finish but Still Fail: Reproducible Triggers and Trace Diagnostics for Parallel Web Exploration
- 2026Where Did It Go Wrong? Process-Level Evaluation of Web Agents with Semantic State Tracking
- 2025Mind2Web 2: Evaluating Agentic Search with Agent-as-a-Judge
- 2026LiveBrowseComp: Are Search Agents Searching, or Just Verifying What They Already Know?
- 2026HANSEL: Extracting Breadcrumbs from Web Agent Trajectories for Interactive Verification
- 2026EviGraph: Towards Verifiable Evidence Construction for Information-Seeking Agents