# MedBrowseComp: Benchmarking Medical Deep Research and Computer Use

arXiv: https://arxiv.org/abs/2505.14963 · HTML: https://mlsanity.com/paper/2505.14963/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 32 references that are in the map.
- First posted on arXiv 20 May 2025.

## Builds on (most cited first, 24 of 24)

- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [Measuring Mathematical Problem Solving With the MATH Dataset](https://mlsanity.com/paper/2103.03874/) (2021)
- [Voyager: An Open-Ended Embodied Agent with Large Language Models](https://mlsanity.com/paper/2305.16291/) (2023)
- [What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Exams](https://mlsanity.com/paper/2009.13081/) (2020)
- [Mind2Web: Towards a Generalist Agent for the Web](https://mlsanity.com/paper/2306.06070/) (2023)
- [OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments](https://mlsanity.com/paper/2404.07972/) (2024)
- [GAIA: a benchmark for General AI Assistants](https://mlsanity.com/paper/2311.12983/) (2023)
- [BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agents](https://mlsanity.com/paper/2504.12516/) (2025)
- [UI-TARS: Pioneering Automated GUI Interaction with Native Agents](https://mlsanity.com/paper/2501.12326/) (2025)
- [Measuring short-form factuality in large language models](https://mlsanity.com/paper/2411.04368/) (2024)
- [LegalBench: A Collaboratively Built Benchmark for Measuring Legal Reasoning in Large Language Models](https://mlsanity.com/paper/2308.11462/) (2023)
- [FreshLLMs: Refreshing Large Language Models with Search Engine Augmentation](https://mlsanity.com/paper/2310.03214/) (2023)
- [Aguvis: Unified Pure Vision Agents for Autonomous GUI Interaction](https://mlsanity.com/paper/2412.04454/) (2024)
- [WebWalker: Benchmarking LLMs in Web Traversal](https://mlsanity.com/paper/2501.07572/) (2025)
- [Fact, Fetch, and Reason: A Unified Evaluation of Retrieval-Augmented Generation](https://mlsanity.com/paper/2409.12941/) (2024)
- [Android in the Zoo: Chain-of-Action-Thought for GUI Agents](https://mlsanity.com/paper/2403.02713/) (2024)
- [BrowseComp-ZH: Benchmarking Web Browsing Ability of Large Language Models in Chinese](https://mlsanity.com/paper/2504.19314/) (2025)
- [Medical Hallucinations in Foundation Models and Their Impact on Healthcare](https://mlsanity.com/paper/2503.05777/) (2025)
- [Benchmarking Large Language Models on CMExam -- A Comprehensive Chinese Medical Exam Dataset](https://mlsanity.com/paper/2306.03030/) (2023)
- [Explorer: Scaling Exploration-driven Web Trajectory Synthesis for Multimodal Web Agents](https://mlsanity.com/paper/2502.11357/) (2025)
- [Open Deep Search: Democratizing Search with Open-source Reasoning Agents](https://mlsanity.com/paper/2503.20201/) (2025)
- [BEARCUBS: A benchmark for computer-using web agents](https://mlsanity.com/paper/2503.07919/) (2025)
- [The impact of responding to patient messages with large language model assistance](https://mlsanity.com/paper/2310.17703/) (2023)

## Built on by (most cited first, 30 of 30)

- [BrowseComp-Plus: A More Fair and Transparent Evaluation Benchmark of Deep-Research Agent](https://mlsanity.com/paper/2508.06600/) (2025)
- [Deep Research: A Survey of Autonomous Research Agents](https://mlsanity.com/paper/2508.12752/) (2025)
- [A Survey of LLM-based Deep Search Agents: Paradigm, Optimization, Evaluation, and Challenges](https://mlsanity.com/paper/2508.05668/) (2025)
- [A Survey of Agentic Reasoning for Large Language Models: Towards Recursively Self-Improving and Collective Agents](https://mlsanity.com/paper/2601.12538/) (2026)
- [MiroThinker-1.7 & H1: Towards Heavy-Duty Research Agents via Verification](https://mlsanity.com/paper/2603.15726/) (2026)
- [MedResearcher-R1: Expert-Level Medical Deep Researcher via A Knowledge-Informed Trajectory Synthesis Framework](https://mlsanity.com/paper/2508.14880/) (2025)
- [A Comprehensive Survey on Reinforcement Learning-based Agentic Search: Foundations, Roles, Optimizations, Evaluations, and Applications](https://mlsanity.com/paper/2510.16724/) (2025)
- [DRACO: a Cross-Domain Benchmark for Deep Research Accuracy, Completeness, and Objectivity](https://mlsanity.com/paper/2602.11685/) (2026)
- [From Question Answering to Task Completion: A Survey on Agent System and Harness Design](https://mlsanity.com/paper/2606.20683/) (2026)
- [Agentic Environment Engineering for Large Language Models: A Survey of Environment Modeling, Synthesis, Evaluation, and Application](https://mlsanity.com/paper/2606.12191/) (2026)
- [DEEPMED: Building a Medical DeepResearch Agent via Multi-hop Med-Search Data and Turn-Controlled Agentic Training & Inference](https://mlsanity.com/paper/2601.18496/) (2026)
- [MedXIAOHE: A Comprehensive Recipe for Building Medical MLLMs](https://mlsanity.com/paper/2602.12705/) (2026)
- [HSCodeComp: A Realistic and Expert-level Benchmark for Deep Search Agents in Hierarchical Rule Application](https://mlsanity.com/paper/2510.19631/) (2025)
- [Hybrid Deep Searcher: Scalable Parallel and Sequential Search Reasoning](https://mlsanity.com/paper/2508.19113/) (2025)
- [AgentEHR: Advancing Autonomous Clinical Decision-Making via Retrospective Summarization](https://mlsanity.com/paper/2601.13918/) (2026)
- [Can Deep Research Agents Retrieve and Organize? Evaluating the Synthesis Gap with Expert Taxonomies](https://mlsanity.com/paper/2601.12369/) (2026)
- [The Path to Self-Evolving Clinical Systems: Scaling Medical Agents from Assistance to Autonomy](https://mlsanity.com/paper/2607.11175/) (2026)
- [Infherno: End-to-end Agent-based FHIR Resource Synthesis from Free-form Clinical Notes](https://mlsanity.com/paper/2507.12261/) (2025)
- [InterDeepResearch: Enabling Human-Agent Collaborative Information Seeking through Interactive Deep Research](https://mlsanity.com/paper/2603.12608/) (2026)
- [MedProbeBench: Systematic Benchmarking at Deep Evidence Integration for Expert-level Medical Guideline](https://mlsanity.com/paper/2604.18418/) (2026)
- [Delegation Intelligence in Deep Search: A Controllable Framework for Disentangled Capability Diagnosis](https://mlsanity.com/paper/2607.23524/) (2026)
- [WANDR: A Benchmark for Wide and Deep Research](https://mlsanity.com/paper/2608.14747/) (2026)
- [MedZERO: Self-Evolving Agents for Open-Ended Medical Reasoning Through Controlled Knowledge Accumulation](https://mlsanity.com/paper/2610.08327/) (2026)
- [DETOUR: An Interactive Benchmark for Dual-Agent Search and Reasoning](https://mlsanity.com/paper/2602.00352/) (2026)
- [Multimodal QUD: Inquisitive Questions from Scientific Figures](https://mlsanity.com/paper/2604.23733/) (2026)
- [MultiDx: A Multi-Source Knowledge Integration Framework towards Diagnostic Reasoning](https://mlsanity.com/paper/2604.24186/) (2026)
- [What Makes a Medical Checker Trainable? Diagnosing Signal Collapse and Reward Hacking in Checker-Guided RAG for Biomedical QA](https://mlsanity.com/paper/2605.25988/) (2026)
- [VaseMuseum: Digital Intelligent Museum for Ancient Greek Pottery](https://mlsanity.com/paper/2607.06374/) (2026)
- [Benchmarking Hybrid Deep Research Across Database Querying and Web Search](https://mlsanity.com/paper/2609.09410/) (2026)
- [LifeSearch: Benchmarking Deep Research Agents for Real-World Life and Productivity](https://mlsanity.com/paper/2610.18092/) (2026)

## Abstract (arXiv)

Large language models (LLMs) are increasingly envisioned as decision-support tools in clinical practice, yet safe clinical reasoning demands integrating heterogeneous knowledge bases -- trials, primary studies, regulatory documents, and cost data -- under strict accuracy constraints. Existing evaluations often rely on synthetic prompts, reduce the task to single-hop factoid queries, or conflate reasoning with open-ended generation, leaving their real-world utility unclear. To close this gap, we present MedBrowseComp, the first benchmark that systematically tests an agent's ability to reliably retrieve and synthesize multi-hop medical facts from live, domain-specific knowledge bases. MedBrowseComp contains more than 1,000 human-curated questions that mirror clinical scenarios where practitioners must reconcile fragmented or conflicting information to reach an up-to-date conclusion. Applying MedBrowseComp to frontier agentic systems reveals performance shortfalls as low as ten percent, exposing a critical gap between current LLM capabilities and the rigor demanded in clinical settings. MedBrowseComp therefore offers a clear testbed for reliable medical information seeking and sets concrete goals for future model and toolchain upgrades. You can visit our project page at: https://moreirap12.github.io/mbc-browse-app/