Towards Personalized Deep Research: Benchmarks and Evaluations
On the map: LLM agents and tool use → Web, search, and research agents
Abstract (arXiv)
Deep Research Agents (DRAs) can autonomously conduct complex investigations and generate comprehensive reports, demonstrating strong real-world potential. However, existing evaluations mostly rely on close-ended benchmarks, while open-ended deep research benchmarks remain scarce and typically neglect personalized scenarios. To bridge this gap, we introduce Personalized Deep Research Bench (PDR-Bench), the first benchmark for evaluating personalization in DRAs. It pairs 50 diverse research tasks across 10 domains with 25 authentic user profiles that combine structured persona attributes with dynamic real-world contexts, yielding 250 realistic user-task queries. To assess system performance, we propose the PQR Evaluation Framework, which jointly measures Personalization Alignment, Content Quality, and Factual Reliability. Our experiments on a range of systems highlight current capabilities and limitations in handling personalized deep research. This work establishes a rigorous foundation for developing and evaluating the next generation of truly personalized AI research assistants.
Key facts. Cited by 16 papers in the map (data through 14 Oct 2026). Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use. Builds on 26 of its 47 references that are in the map. First posted on arXiv 29 Sep 2025.
Builds on
26 of 47 references.
Closest ancestors · 4
Same direction, newest first
- 2025BrowseComp-Plus: A More Fair and Transparent Evaluation Benchmark of Deep-Research Agent8 Aug
- 2025DeepResearch Bench: A Comprehensive Benchmark for Deep Research Agents13 Jun
- 2025DeepResearchGym: A Free, Transparent, and Reproducible Evaluation Sandbox for Deep Research25 May
- 2025Search-o1: Agentic Search-Enhanced Large Reasoning Models9 Jan
Other references · 22
- 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,282
- 2023Large Language Modelscited by 780
- 2025Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memorycited by 779
- 2025BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agentscited by 647
- 2025WebThinker: Empowering Large Reasoning Models with Deep Research Capabilitycited by 335
- 2023FacTool: Factuality Detection in Generative AI -- A Tool Augmented Framework for Multi-Task and Multi-Domain Scenarioscited by 251
- 2025Know Me, Respond to Me: Benchmarking LLMs for Dynamic User Profiling and Personalized Responses at Scalecited by 178
- 2024Long-form factuality in large language modelscited by 155
- 2025OWL: Optimized Workforce Learning for General Multi-Agent Assistance in Real-World Task Automationcited by 131
- 2023Agents: An Open-source Framework for Autonomous Language Agentscited by 116
- 2024Symbolic Learning Enables Self-Evolving Agentscited by 106
- 2025Agent KB: Leveraging Cross-Domain Experience for Agentic Problem Solvingcited by 94
10 more
- 2025Chain-of-Agents: End-to-End Agent Foundation Models via Multi-Agent Distillation and Agentic RLcited by 80
- 2025Scaling Test-time Compute for LLM Agentscited by 63
- 2025TaskCraft: Automated Generation of Agentic Taskscited by 51
- 2025PersonaLens: A Benchmark for Personalization Evaluation in Conversational AI Assistantscited by 48
- 2025A Survey on Personalized Alignment -- The Missing Piece for Large Language Models in Real-World Applicationscited by 42
- 2024Personality Alignment of Large Language Modelscited by 36
- 2024AI PERSONA: Towards Life-long Personalization of LLMscited by 32
- 2025Efficient Agents: Building Effective Agents While Reducing Costcited by 32
- 2023Learning Personalized Alignment for Evaluating Open-ended Text Generationcited by 29
- 2025PersonaFeedback: A Large-scale Human-annotated Benchmark For Personalizationcited by 15
Built on by
Citing papers per year: 2025: 1; 2026: 15.
16 papers.
Sort:
By area:
- 2025O-Mem: Omni Memory System for Personalized, Long Horizon, Self-Evolving Agentscited by 32
- 2026Toward Personalized LLM-Powered Agents: Foundations, Evaluation, and Future Directionscited by 14
- 2026O-Researcher: An Open Ended Deep Research Model via Multi-Agent Distillation and Agentic RLcited by 11
- 2026MiroEval: Benchmarking Multimodal Deep Research Agents in Process and Outcomecited by 6
- 2026Why Your Deep Research Agent Fails? On Hallucination Evaluation in Full Research Trajectorycited by 5
- 2026R2-Write: Reflection and Revision for Open-Ended Writing with Deep Reasoningcited by 3
- 2026IntentRL: Training Proactive User-intent Agents for Open-ended Deep Research via Reinforcement Learningcited by 2
- 2026EcoGym: Evaluating LLMs for Long-Horizon Plan-and-Execute in Interactive Economiescited by 2
- 2026Language Models Don't Know What You Want: Evaluating Personalization in Deep Research Needs Real Userscited by 2
- 2026Towards Knowledgeable Deep Research: Framework and Benchmarkcited by 1
- 2026DRACULA: Hunting for the Actions Users Want Deep Research Agents to Executecited by 1
- 2026Personalized Deep Research Query Refinement with Graph-Scaffolded Evidence Groundingcited by 1
4 more
- 2026MobileMem: Learning from a Year of Mobile Experiencescited by 1
- 2026DeepSynth-Eval: Objectively Evaluating Information Consolidation in Deep Survey Writingcited by 0
- 2026PaperFlow: Profiling, Recommending, and Adapting Across Daily Paper Streamscited by 0
- 2026What Happens During Autonomous Deep Research After the User Steps Away?cited by 0
- 2026What Happens During Autonomous Deep Research After the User Steps Away?27 Sep 2026
- 2026MobileMem: Learning from a Year of Mobile Experiences11 Aug 2026
- 2026Personalized Deep Research Query Refinement with Graph-Scaffolded Evidence Grounding6 Aug 2026
- 2026PaperFlow: Profiling, Recommending, and Adapting Across Daily Paper Streams5 Jun 2026
- 2026DRACULA: Hunting for the Actions Users Want Deep Research Agents to Execute26 Apr 2026
- 2026Towards Knowledgeable Deep Research: Framework and Benchmark9 Apr 2026
- 2026R2-Write: Reflection and Revision for Open-Ended Writing with Deep Reasoning3 Apr 2026
- 2026MiroEval: Benchmarking Multimodal Deep Research Agents in Process and Outcome30 Mar 2026
- 2026Language Models Don't Know What You Want: Evaluating Personalization in Deep Research Needs Real Users17 Mar 2026
- 2026Toward Personalized LLM-Powered Agents: Foundations, Evaluation, and Future Directions26 Feb 2026
- 2026EcoGym: Evaluating LLMs for Long-Horizon Plan-and-Execute in Interactive Economies10 Feb 2026
- 2026IntentRL: Training Proactive User-intent Agents for Open-ended Deep Research via Reinforcement Learning3 Feb 2026
4 more
- 2026Why Your Deep Research Agent Fails? On Hallucination Evaluation in Full Research Trajectory30 Jan 2026
- 2026O-Researcher: An Open Ended Deep Research Model via Multi-Agent Distillation and Agentic RL7 Jan 2026
- 2026DeepSynth-Eval: Objectively Evaluating Information Consolidation in Deep Survey Writing7 Jan 2026
- 2025O-Mem: Omni Memory System for Personalized, Long Horizon, Self-Evolving Agents17 Nov 2025
- 2025O-Mem: Omni Memory System for Personalized, Long Horizon, Self-Evolving Agentscited by 32
- 2026Toward Personalized LLM-Powered Agents: Foundations, Evaluation, and Future Directionscited by 14
- 2026O-Researcher: An Open Ended Deep Research Model via Multi-Agent Distillation and Agentic RLcited by 11
- 2026MiroEval: Benchmarking Multimodal Deep Research Agents in Process and Outcomecited by 6
- 2026Why Your Deep Research Agent Fails? On Hallucination Evaluation in Full Research Trajectorycited by 5
- 2026IntentRL: Training Proactive User-intent Agents for Open-ended Deep Research via Reinforcement Learningcited by 2
- 2026EcoGym: Evaluating LLMs for Long-Horizon Plan-and-Execute in Interactive Economiescited by 2
- 2026Towards Knowledgeable Deep Research: Framework and Benchmarkcited by 1
- 2026DRACULA: Hunting for the Actions Users Want Deep Research Agents to Executecited by 1
- 2026Personalized Deep Research Query Refinement with Graph-Scaffolded Evidence Groundingcited by 1
- 2026Personalized Deep Research Query Refinement with Graph-Scaffolded Evidence Grounding6 Aug 2026
- 2026DRACULA: Hunting for the Actions Users Want Deep Research Agents to Execute26 Apr 2026
- 2026Towards Knowledgeable Deep Research: Framework and Benchmark9 Apr 2026
- 2026MiroEval: Benchmarking Multimodal Deep Research Agents in Process and Outcome30 Mar 2026
- 2026Toward Personalized LLM-Powered Agents: Foundations, Evaluation, and Future Directions26 Feb 2026
- 2026EcoGym: Evaluating LLMs for Long-Horizon Plan-and-Execute in Interactive Economies10 Feb 2026
- 2026IntentRL: Training Proactive User-intent Agents for Open-ended Deep Research via Reinforcement Learning3 Feb 2026
- 2026Why Your Deep Research Agent Fails? On Hallucination Evaluation in Full Research Trajectory30 Jan 2026
- 2026O-Researcher: An Open Ended Deep Research Model via Multi-Agent Distillation and Agentic RL7 Jan 2026
- 2025O-Mem: Omni Memory System for Personalized, Long Horizon, Self-Evolving Agents17 Nov 2025
- 2026Language Models Don't Know What You Want: Evaluating Personalization in Deep Research Needs Real Userscited by 2
- 2026MobileMem: Learning from a Year of Mobile Experiencescited by 1
- 2026DeepSynth-Eval: Objectively Evaluating Information Consolidation in Deep Survey Writingcited by 0
- 2026PaperFlow: Profiling, Recommending, and Adapting Across Daily Paper Streamscited by 0
- 2026What Happens During Autonomous Deep Research After the User Steps Away?cited by 0
- 2026What Happens During Autonomous Deep Research After the User Steps Away?27 Sep 2026
- 2026MobileMem: Learning from a Year of Mobile Experiences11 Aug 2026
- 2026PaperFlow: Profiling, Recommending, and Adapting Across Daily Paper Streams5 Jun 2026
- 2026Language Models Don't Know What You Want: Evaluating Personalization in Deep Research Needs Real Users17 Mar 2026
- 2026DeepSynth-Eval: Objectively Evaluating Information Consolidation in Deep Survey Writing7 Jan 2026
Similar, not cited
Close in wording, no citation either way
- 2026Personalized Deep Research: A User-Centric Framework, Dataset, and Hybrid Evaluation for Knowledge Discovery
- 2026DeepResearch Bench II: Diagnosing Deep Research Agents via Rubrics from Expert Reports
- 2025Dr. Bench: A Multidimensional Evaluation for Deep Research Agents, from Answers to Reports
- 2026LifeSearch: Benchmarking Deep Research Agents for Real-World Life and Productivity
- 2025Deep Research Comparator: A Platform For Fine-grained Human Annotations of Deep Research Agents
- 2025ADRA-Bank: A Modular Benchmark for Academic Deep Research Agents
- 2025ResearchRubrics: A Benchmark of Prompts and Rubrics For Evaluating Deep Research Agents
- 2026DeepResearchEval: An Automated Framework for Deep Research Task Construction and Agentic Evaluation