mlsanityData to 7 Oct 2026

Redesigning and Auditing Deep Research Writing for Faithful Reports

arXiv 2608.28643 ↗First posted 12 Aug 2026cs.CL cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Rubric-based evaluations of deep-research (DR) systems often obscure fine-grained factual failures in generated reports. We introduce CLAIMPROBE, a claim-level audit that decomposes DR reports into claims and measures hallucination, misattribution, citation hygiene, and necessary-fact recall against retrieved evidence. Using CLAIMPROBE, we find that strong DR pipelines can omit key evidence and misattribute claims even when their rubric scores remain stable. We then propose CLAIMWRITER, a hierarchical claim-based writer that extracts source facts, maps them to a query-derived outline, and drafts each section from a source-linked claim representation. Across three prior DR frameworks, replacing only the report writer with CLAIMWRITER reduces hallucination by 2.6 to 4.5 times and improves necessary-fact recall by 1.2 to 1.7 times, while largely preserving overall report quality. CLAIMWRITER also enables localized revision: when sources change, it propagates changed source facts into revised reports at the highest rate among update methods, while also being more cost-effective.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 16 of its 23 references that are in the map. First posted on arXiv 12 Aug 2026.

Builds on

16 of 23 references.

References · 16

  1. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  2. 2022Survey of Hallucination in Natural Language Generationcited by 2,056
  3. 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,259
  4. 2023GAIA: a benchmark for General AI Assistantscited by 1,084
  5. 2023Enabling Large Language Models to Generate Text with Citationscited by 584
  6. 2023Ragas: Automated Evaluation of Retrieval Augmented Generationcited by 408
  7. 2023Evaluating Verifiability in Generative Search Enginescited by 297
  8. 2025DeepResearch Bench: A Comprehensive Benchmark for Deep Research Agentscited by 236
  9. 2024Long-form factuality in large language modelscited by 146
  10. 2022Attributed Question Answering: Evaluation and Modeling for Attributed Large Language Modelscited by 108
  11. 2025ResearchRubrics: A Benchmark of Prompts and Rubrics For Evaluating Deep Research Agentscited by 80
  12. 2025LiveResearchBench: A Live Benchmark for User-Centric Deep Research in the Wildcited by 35
4 more
  1. 2026DeepResearch Bench II: Diagnosing Deep Research Agents via Rubrics from Expert Reportscited by 33
  2. 2024Search Engines in an AI Era: The False Promise of Factual and Verifiable Source-Cited Responsescited by 15
  3. 2025DeepTRACE: Auditing Deep Research AI Systems for Tracking Reliability Across Citations and Evidencecited by 10
  4. 2026Don't Stop Early: Scalable Enterprise Deep Research with Controlled Information Flow and Evidence-Aware Terminationcited by 4

Built on by

Not cited yet.