When Web Agents Finish but Still Fail: Reproducible Triggers and Trace Diagnostics for Parallel Web Exploration
On the map: LLM agents and tool use → Web, search, and research agents
Abstract (arXiv)
Long-horizon web agents often fail in ways hidden by final-answer evaluation: they may visit useful pages, produce a well-formed answer, and terminate confidently while still missing fields, over-including unsupported items, or relying on stale evidence. We study these failures with Parallel WebBench, a parallel web-exploration benchmark containing 1,679 verified records: 350 manually curated parallel tasks and 1,329 reconstructed records with verified URL-based trajectories. We train WebExplorer-style agents with GRPO under human-only, balanced human-synthetic, and synthetic-heavy data mixtures. At 16k context and 16 interaction rounds, the best GRPO model improves completion over WebExplorer-8B from 50.7% to 96.0% and GPT-4.1-mini-judged element-wise F1 from 0.2489 to 0.4529, but binary accuracy remains far below completion. Trace-level analysis identifies three persistent failure modes: context-bound search loops, premature termination on partial answers, and synthesis collapse after relevant evidence has already been retrieved. These results show that synthetic-data GRPO reduces abstention and improves partial correctness, but leaves a completion-correctness gap that requires evidence-grounded coverage and synthesis diagnostics.
Key facts. Cited by 1 paper in the map (data through 13 Oct 2026). Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use. Builds on 34 of its 41 references that are in the map. First posted on arXiv 16 Jun 2026.
Builds on
34 of 41 references.
Closest ancestors · 6
Same direction, newest first
- 2025WebExplorer: Explore and Evolve for Training Long-Horizon Web Agents8 Sep
- 2024WorkArena: How Capable Are Web Agents at Solving Common Knowledge Work Tasks?12 Mar
- 2024GPT-4V(ision) is a Generalist Web Agent, if Grounded3 Jan
- 2023WebArena: A Realistic Web Environment for Building Autonomous Agents25 Jul
- 2023Mind2Web: Towards a Generalist Agent for the Web9 Jun
- 2022WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agents4 Jul
Other references · 28
- 2022Training language models to follow instructions with human feedbackcited by 15,856
- 2017Proximal Policy Optimization Algorithmscited by 14,592
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,221
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,091
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,502
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,492
- 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,156
- 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,117
- 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,797
- 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,716
- 2023Let's Verify Step by Stepcited by 3,672
- 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,181
16 more
- 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,228
- 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,736
- 2016Concrete Problems in AI Safetycited by 1,614
- 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,368
- 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,268
- 2023AgentBench: Evaluating LLMs as Agentscited by 1,242
- 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environmentscited by 1,154
- 2023Large Language Models are not Fair Evaluatorscited by 1,054
- 2025BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agentscited by 628
- 2023Enabling Large Language Models to Generate Text with Citationscited by 588
- 2020KILT: a Benchmark for Knowledge Intensive Language Taskscited by 571
- 2023Prometheus: Inducing Fine-grained Evaluation Capability in Language Modelscited by 563
- 2023FreshLLMs: Refreshing Large Language Models with Search Engine Augmentationcited by 312
- 2021Measuring Attribution in Natural Language Generation Modelscited by 253
- 2024WebLINX: Real-World Website Navigation with Multi-Turn Dialoguecited by 179
- 2025WebWalker: Benchmarking LLMs in Web Traversalcited by 173
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026Odysseys: Benchmarking Web Agents on Realistic Long Horizon Tasks
- 2025WebArxiv: A Reproducible Benchmark for Evaluating Multimodal Web Agents on arXiv Tasks
- 2024WebSuite: Systematically Evaluating Why Web Agents Fail
- 2025BrowserArena: Evaluating LLM Agents on Real-World Web Navigation Tasks
- 2025BEARCUBS: A benchmark for computer-using web agents
- 2026WebRetriever: A Large-Scale Comprehensive Benchmark for Efficient Web Agent Evaluation
- 2026BrowserForge: Scaling Web Episode via Parallel Browser Sandboxes
- 2026WebChallenger: A Reliable and Efficient Generalist Web Agent