mlsanityData to 14 Oct 2026

BrowseComp-ZH: Benchmarking Web Browsing Ability of Large Language Models in Chinese

arXiv 2504.19314 ↗First posted 27 Apr 2025cs.CLcited by 145

Not on the map. Why?

Cited by 97 in LLM agents and tool use, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

As large language models (LLMs) evolve into tool-using agents, the ability to browse the web in real-time has become a critical yardstick for measuring their reasoning and retrieval competence. Existing benchmarks such as BrowseComp concentrate on English and overlook the linguistic, infrastructural, and censorship-related complexities of other major information ecosystems -- most notably Chinese. To address this gap, we introduce BrowseComp-ZH, a high-difficulty benchmark purpose-built to comprehensively evaluate LLM agents on the Chinese web. BrowseComp-ZH consists of 289 multi-hop questions spanning 11 diverse domains. Each question is reverse-engineered from a short, objective, and easily verifiable answer (e.g., a date, number, or proper noun). A two-stage quality control protocol is applied to strive for high question difficulty and answer uniqueness. We benchmark over 20 state-of-the-art language models and agentic search systems on our proposed BrowseComp-ZH. Despite their strong conversational and retrieval capabilities, most models struggle severely: a large number achieve accuracy rates below 10%, and only a handful exceed 20%. Even the best-performing system, OpenAI's DeepResearch, reaches just 42.9%. These results demonstrate the considerable difficulty of BrowseComp-ZH, where success demands not only effective retrieval strategies, but also sophisticated reasoning and information reconciliation -- capabilities that current models still struggle to master. Our dataset, construction guidelines, and benchmark results have been publicly released at https://github.com/PALIN2018/BrowseComp-ZH.

Key facts. Cited by 145 papers in the map (data through 14 Oct 2026). Builds on 21 of its 34 references that are in the map. First posted on arXiv 27 Apr 2025.

Builds on

21 of 34 references.

References · 21

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  2. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  3. 2024Qwen2.5 Technical Reportcited by 4,942
  4. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
  5. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
  6. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,955
  7. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,141
  8. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,745
  9. 2024From Local to Global: A Graph RAG Approach to Query-Focused Summarizationcited by 1,453
  10. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,439
  11. 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,375
  12. 2023GAIA: a benchmark for General AI Assistantscited by 1,128
9 more
  1. 2025BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agentscited by 647
  2. 2020KILT: a Benchmark for Knowledge Intensive Language Taskscited by 573
  3. 2023FreshLLMs: Refreshing Large Language Models with Search Engine Augmentationcited by 318
  4. 2024AutoWebGLM: A Large Language Model-based Web Navigating Agentcited by 130
  5. 2023Benchmarking Large Language Models on CMExam -- A Comprehensive Chinese Medical Exam Datasetcited by 91
  6. 2024CBR-RAG: Case-Based Reasoning for Retrieval Augmented Generation in LLMs for Legal Question Answeringcited by 72
  7. 2024When Search Engine Services meet Large Language Models: Visions and Challengescited by 59
  8. 2024Let LLMs Take on the Latest Challenges! A Chinese Dynamic Question Answering Benchmarkcited by 22
  9. 2024Level-Navi Agent: A Framework and benchmark for Chinese Web Search Agentscited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 55; 2026: 90.

145 papers.

Sort:

By area:

  1. 2026GLM-5: from Vibe Coding to Agentic Engineeringcited by 587
  2. 2025DeepResearch Bench: A Comprehensive Benchmark for Deep Research Agentscited by 248
  3. 2025WebSailor: Navigating Super-human Reasoning for Web Agentcited by 229
  4. 2025UI-TARS-2 Technical Report: Advancing GUI Agent with Multi-Turn Reinforcement Learningcited by 215
  5. 2025BrowseComp-Plus: A More Fair and Transparent Evaluation Benchmark of Deep-Research Agentcited by 200
  6. 2025WebDancer: Towards Autonomous Information Seeking Agencycited by 188
  7. 2025ReSum: Unlocking Long-Horizon Search Intelligence via Context Summarizationcited by 141
  8. 2025WebShaper: Agentically Data Synthesizing via Information-Seeking Formalizationcited by 121
  9. 2025AgentFold: Long-Horizon Web Agents with Proactive Context Managementcited by 103
  10. 2025MiroThinker: Pushing the Performance Boundaries of Open-Source Research Agents via Model, Context, and Interactive Scalingcited by 81
  11. 2025WideSearch: Benchmarking Agentic Broad Info-Seekingcited by 69
  12. 2025WebResearcher: Unleashing unbounded reasoning capability in Long-Horizon Agentscited by 68
38 more
  1. 2025Deep Research: A Survey of Autonomous Research Agentscited by 66
  2. 2025Tongyi DeepResearch Technical Reportcited by 66
  3. 2025WebSailor-V2: Bridging the Chasm to Proprietary Agents via Synthetic Data and Scalable Reinforcement Learningcited by 61
  4. 2025SimpleDeepSearcher: Deep Information Seeking via Web-Powered Reasoning Trajectory Synthesiscited by 55
  5. 2025DeepDive: Advancing Deep Search Agents with Knowledge Graphs and Multi-Turn RLcited by 55
  6. 2025From Web Search towards Agentic Deep Research: Incentivizing Search with Reasoning Agentscited by 51
  7. 2026A Survey of Agentic Reasoning for Large Language Models: Towards Recursively Self-Improving and Collective Agentscited by 50
  8. 2026Seed1.8 Model Card: Towards Generalized Real-World Agencycited by 50
  9. 2026REDSearcher: A Scalable and Cost-Efficient Framework for Long-Horizon Search Agentscited by 49
  10. 2025Budget-Aware Tool Use Enables Effective Agent Scalingcited by 44
  11. 2025Towards Agentic RAG with Deep Reasoning: A Survey of RAG-Reasoning Systems in LLMscited by 42
  12. 2025MM-BrowseComp: A Comprehensive Benchmark for Multimodal Browsing Agentscited by 42
  13. 2025Scaling Agents via Continual Pre-trainingcited by 40
  14. 2026OpenSeeker: Democratizing Frontier Search Agents by Fully Open-Sourcing Training Datacited by 39
  15. 2025ReportBench: Evaluating Deep Research Agents via Academic Survey Taskscited by 38
  16. 2026OpenResearcher: A Fully Open Pipeline for Long-Horizon Deep Research Trajectory Synthesiscited by 37
  17. 2026LongCat-Flash-Thinking-2601 Technical Reportcited by 34
  18. 2026MiroThinker-1.7 & H1: Towards Heavy-Duty Research Agents via Verificationcited by 32
  19. 2025MedBrowseComp: Benchmarking Medical Deep Research and Computer Usecited by 30
  20. 2025Let It Flow: Agentic Crafting on Rock and Roll, Building the ROME Model within an Open Agentic Learning Ecosystemcited by 30
  21. 2025FinSearchComp: Towards a Realistic, Expert-Level Evaluation of Financial Search and Reasoningcited by 27
  22. 2025IterResearch: Rethinking Long-Horizon Agents with Interaction Scalingcited by 23
  23. 2025A Comprehensive Survey on Reinforcement Learning-based Agentic Search: Foundations, Roles, Optimizations, Evaluations, and Applicationscited by 22
  24. 2025DeepWideSearch: Benchmarking Depth and Width in Agentic Information Seekingcited by 21
  25. 2025Beyond Turn Limits: Training Deep Search Agents with Dynamic Context Windowcited by 19
  26. 2026Learning Query-Specific Rubrics from Human Preferences for DeepResearch Report Generationcited by 18
  27. 2025BrowseMaster: Towards Scalable Web Browsing via Tool-Augmented Programmatic Agent Paircited by 16
  28. 2026The Confidence Dichotomy: Analyzing and Mitigating Miscalibration in Tool-Use Agentscited by 16
  29. 2026Marco DeepResearch: Unlocking Efficient Deep Research Agents via Verification-Centric Designcited by 16
  30. 2025InfoDeepSeek: Benchmarking Agentic Information Seeking for Retrieval-Augmented Generationcited by 15
  31. 2025From Code Foundation Models to Agents and Applications: A Comprehensive Survey and Practical Guide to Code Intelligencecited by 15
  32. 2026MiroFlow: Towards High-Performance and Robust Open-Source Agent Framework for General Deep Research Taskscited by 15
  33. 2025Reinforcement Learning Foundations for Deep Research Systems: A Surveycited by 14
  34. 2025ParallelMuse: Agentic Parallel Thinking for Deep Information Seekingcited by 14
  35. 2026Chaining the Evidence: Robust Reinforcement Learning for Deep Search Agents with Citation-Aware Rubric Rewardscited by 14
  36. 2025Pushing Test-Time Scaling Limits of Deep Search with Asymmetric Verificationcited by 12
  37. 2025InteractComp: Evaluating Search Agents With Ambiguous Queriescited by 12
  38. 2026OffSeeker: Online Reinforcement Learning Is Not All You Need for Deep Research Agentscited by 12

50 most cited of 145 All 145 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Projecting BrowseComp-Plus onto ClimbMix: Toward More Realistic Corpora for Agentic Search
  2. 2026BrowseComp-V3: A Visual, Vertical, and Verifiable Benchmark for Multimodal Browsing Agents
  3. 2026EvoBrowseComp: Benchmarking Search Agents on Evolving Knowledge
  4. 2026LiveNewsBench: Evaluating LLM Web Search Capabilities with Freshly Curated News
  5. 2025ChineseEcomQA: A Scalable E-commerce Concept Evaluation Benchmark for Large Language Models
  6. 2025PRISM: Agentic Retrieval with LLMs for Multi-Hop Question Answering
  7. 2025ManuSearch: Democratizing Deep Search in Large Language Models with a Transparent and Open Multi-Agent Framework
  8. 2026Q2D-Web: A Large-Scale Benchmark for Retrieval in Agentic RAG Systems