# Why Your Deep Research Agent Fails? On Hallucination Evaluation in Full Research Trajectory

arXiv: https://arxiv.org/abs/2601.22984 · HTML: https://mlsanity.com/paper/2601.22984/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in Medical and scientific domain agents, one of 16 directions in LLM agents and tool use.
- Builds on 39 of its 77 references that are in the map.
- First posted on arXiv 30 Jan 2026.

## Placed in

- LLM agents and tool use / [Medical and scientific domain agents](https://mlsanity.com/agents/medical-and-scientific-domain-agents/)

## Builds on (most cited first, 30 of 39)

- [Lost in the Middle: How Language Models Use Long Contexts](https://mlsanity.com/paper/2307.03172/) (2023)
- [FEVER: a large-scale dataset for Fact Extraction and VERification](https://mlsanity.com/paper/1803.05355/) (2018)
- [FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation](https://mlsanity.com/paper/2305.14251/) (2023)
- [GAIA: a benchmark for General AI Assistants](https://mlsanity.com/paper/2311.12983/) (2023)
- [Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Models](https://mlsanity.com/paper/2309.01219/) (2023)
- [BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agents](https://mlsanity.com/paper/2504.12516/) (2025)
- [Why Do Multi-Agent LLM Systems Fail?](https://mlsanity.com/paper/2503.13657/) (2025)
- [DeepResearch Bench: A Comprehensive Benchmark for Deep Research Agents](https://mlsanity.com/paper/2506.11763/) (2025)
- [HoVer: A Dataset for Many-Hop Fact Extraction And Claim Verification](https://mlsanity.com/paper/2011.03088/) (2020)
- [BrowseComp-Plus: A More Fair and Transparent Evaluation Benchmark of Deep-Research Agent](https://mlsanity.com/paper/2508.06600/) (2025)
- [Which Agent Causes Task Failures and When? On Automated Failure Attribution of LLM Multi-Agent Systems](https://mlsanity.com/paper/2505.00212/) (2025)
- [Long-form factuality in large language models](https://mlsanity.com/paper/2403.18802/) (2024)
- [Where LLM Agents Fail and How They can Learn From Failures](https://mlsanity.com/paper/2509.25370/) (2025)
- [Deep Research Agents: A Systematic Examination And Roadmap](https://mlsanity.com/paper/2506.18096/) (2025)
- [xbench: Tracking Agents Productivity Scaling with Profession-Aligned Real-World Evaluations](https://mlsanity.com/paper/2506.13651/) (2025)
- [SciFact-Open: Towards open-domain scientific claim verification](https://mlsanity.com/paper/2210.13777/) (2022)
- [Deep Research: A Survey of Autonomous Research Agents](https://mlsanity.com/paper/2508.12752/) (2025)
- [LLM-based Agents Suffer from Hallucinations: A Survey of Taxonomy, Methods, and Directions](https://mlsanity.com/paper/2509.18970/) (2025)
- [RaFe: Ranking Feedback Improves Query Rewriting for RAG](https://mlsanity.com/paper/2405.14431/) (2024)
- [ReportBench: Evaluating Deep Research Agents via Academic Survey Tasks](https://mlsanity.com/paper/2508.15804/) (2025)
- [LiveResearchBench: A Live Benchmark for User-Centric Deep Research in the Wild](https://mlsanity.com/paper/2510.14240/) (2025)
- [DeepScholar-Bench: A Live Benchmark and Automated Evaluation for Generative Research Synthesis](https://mlsanity.com/paper/2508.20033/) (2025)
- [DeepResearch Bench II: Diagnosing Deep Research Agents via Rubrics from Expert Reports](https://mlsanity.com/paper/2601.08536/) (2026)
- [ResearcherBench: Evaluating Deep AI Research Systems on the Frontiers of Scientific Inquiry](https://mlsanity.com/paper/2507.16280/) (2025)
- [DeepResearch Arena: The First Exam of LLMs' Research Abilities via Seminar-Grounded Tasks](https://mlsanity.com/paper/2509.01396/) (2025)
- [Beyond the Final Answer: Evaluating the Reasoning Trajectories of Tool-Augmented Agents](https://mlsanity.com/paper/2510.02837/) (2025)
- [DRBench: A Realistic Benchmark for Enterprise Deep Research](https://mlsanity.com/paper/2510.00172/) (2025)
- [MMDeepResearch-Bench: A Benchmark for Multimodal Deep Research Agents](https://mlsanity.com/paper/2601.12346/) (2026)
- [Atomic Fact Decomposition Helps Attributed Question Answering](https://mlsanity.com/paper/2410.16708/) (2024)
- [Small Agent Can Also Rock! Empowering Small Language Models as Hallucination Detector](https://mlsanity.com/paper/2406.11277/) (2024)

## Built on by (most cited first, 5 of 5)

- [Time to REFLECT: Can We Trust LLM Judges for Evidence-based Research Agents?](https://mlsanity.com/paper/2605.19196/) (2026)
- [How Do Agents Fail on AutoResearch: End-to-End Diagnostic Evaluation on 100 Real-World Frontier Research Tasks](https://mlsanity.com/paper/2608.14905/) (2026)
- [SearchAuditor: Auditing and Attributing Failures in Long-Horizon Search Agents](https://mlsanity.com/paper/2608.05212/) (2026)
- [Tracing the Cascade: A Topology-Aware Evaluation Framework for Scientific Agent Hallucinations](https://mlsanity.com/paper/2608.00711/) (2026)
- [PHRBench: A Behavioral Evaluation of Post-Hallucination Reasoning in LLMs](https://mlsanity.com/paper/2610.10455/) (2026)

## Abstract (arXiv)

Diagnosing failure patterns in Deep Research Agents (DRAs) remains a critical challenge. Existing benchmarks predominantly rely on end-to-end evaluation, obscuring intermediate hallucinations that accumulate throughout the research trajectory. To bridge this gap, we propose a shift from outcome-based to process-aware evaluation by auditing hallucinations in the full plan-search-summarize trajectory. We introduce the PING Taxonomy, which categorizes DRA hallucinations into four complementary types: Propagation, Intent, Noise-induced, and Grounding. We further instantiate this taxonomy into a fine-grained evaluation framework that decomposes trajectories into atomic actions, claims, and sub-queries for rigorous verification, and we validate its reliability on standard fact-checking benchmarks and human-reviewed trajectories. Leveraging this framework to isolate 100 hallucination-prone tasks, including adversarial scenarios, we curate DeepHalluBench. Experiments on six representative DRAs show that, on our hallucination-prone stress-test set, all evaluated systems still exhibit non-negligible reliability gaps. Furthermore, our diagnostic analysis traces these failures to systemic deficits, especially hallucination propagation and cognitive biases, providing actionable insights for future architectural optimization. Code and data are available at https://github.com/yuhao-zhan/DeepHalluBench.