# Towards Personalized Deep Research: Benchmarks and Evaluations

arXiv: https://arxiv.org/abs/2509.25106 · HTML: https://mlsanity.com/paper/2509.25106/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use.
- Builds on 26 of its 47 references that are in the map.
- First posted on arXiv 29 Sep 2025.

## Placed in

- LLM agents and tool use / [Web, search, and research agents](https://mlsanity.com/agents/web-search-and-research-agents/)

## Builds on (most cited first, 26 of 26)

- [FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation](https://mlsanity.com/paper/2305.14251/) (2023)
- [Large Language Models](https://mlsanity.com/paper/2307.05782/) (2023)
- [Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory](https://mlsanity.com/paper/2504.19413/) (2025)
- [BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agents](https://mlsanity.com/paper/2504.12516/) (2025)
- [Search-o1: Agentic Search-Enhanced Large Reasoning Models](https://mlsanity.com/paper/2501.05366/) (2025)
- [WebThinker: Empowering Large Reasoning Models with Deep Research Capability](https://mlsanity.com/paper/2504.21776/) (2025)
- [FacTool: Factuality Detection in Generative AI -- A Tool Augmented Framework for Multi-Task and Multi-Domain Scenarios](https://mlsanity.com/paper/2307.13528/) (2023)
- [DeepResearch Bench: A Comprehensive Benchmark for Deep Research Agents](https://mlsanity.com/paper/2506.11763/) (2025)
- [BrowseComp-Plus: A More Fair and Transparent Evaluation Benchmark of Deep-Research Agent](https://mlsanity.com/paper/2508.06600/) (2025)
- [Know Me, Respond to Me: Benchmarking LLMs for Dynamic User Profiling and Personalized Responses at Scale](https://mlsanity.com/paper/2504.14225/) (2025)
- [Long-form factuality in large language models](https://mlsanity.com/paper/2403.18802/) (2024)
- [OWL: Optimized Workforce Learning for General Multi-Agent Assistance in Real-World Task Automation](https://mlsanity.com/paper/2505.23885/) (2025)
- [Agents: An Open-source Framework for Autonomous Language Agents](https://mlsanity.com/paper/2309.07870/) (2023)
- [Symbolic Learning Enables Self-Evolving Agents](https://mlsanity.com/paper/2406.18532/) (2024)
- [Agent KB: Leveraging Cross-Domain Experience for Agentic Problem Solving](https://mlsanity.com/paper/2507.06229/) (2025)
- [Chain-of-Agents: End-to-End Agent Foundation Models via Multi-Agent Distillation and Agentic RL](https://mlsanity.com/paper/2508.13167/) (2025)
- [Scaling Test-time Compute for LLM Agents](https://mlsanity.com/paper/2506.12928/) (2025)
- [DeepResearchGym: A Free, Transparent, and Reproducible Evaluation Sandbox for Deep Research](https://mlsanity.com/paper/2505.19253/) (2025)
- [TaskCraft: Automated Generation of Agentic Tasks](https://mlsanity.com/paper/2506.10055/) (2025)
- [PersonaLens: A Benchmark for Personalization Evaluation in Conversational AI Assistants](https://mlsanity.com/paper/2506.09902/) (2025)
- [A Survey on Personalized Alignment -- The Missing Piece for Large Language Models in Real-World Applications](https://mlsanity.com/paper/2503.17003/) (2025)
- [Personality Alignment of Large Language Models](https://mlsanity.com/paper/2408.11779/) (2024)
- [AI PERSONA: Towards Life-long Personalization of LLMs](https://mlsanity.com/paper/2412.13103/) (2024)
- [Efficient Agents: Building Effective Agents While Reducing Cost](https://mlsanity.com/paper/2508.02694/) (2025)
- [Learning Personalized Alignment for Evaluating Open-ended Text Generation](https://mlsanity.com/paper/2310.03304/) (2023)
- [PersonaFeedback: A Large-scale Human-annotated Benchmark For Personalization](https://mlsanity.com/paper/2506.12915/) (2025)

## Built on by (most cited first, 16 of 16)

- [O-Mem: Omni Memory System for Personalized, Long Horizon, Self-Evolving Agents](https://mlsanity.com/paper/2511.13593/) (2025)
- [Toward Personalized LLM-Powered Agents: Foundations, Evaluation, and Future Directions](https://mlsanity.com/paper/2602.22680/) (2026)
- [O-Researcher: An Open Ended Deep Research Model via Multi-Agent Distillation and Agentic RL](https://mlsanity.com/paper/2601.03743/) (2026)
- [MiroEval: Benchmarking Multimodal Deep Research Agents in Process and Outcome](https://mlsanity.com/paper/2603.28407/) (2026)
- [Why Your Deep Research Agent Fails? On Hallucination Evaluation in Full Research Trajectory](https://mlsanity.com/paper/2601.22984/) (2026)
- [R2-Write: Reflection and Revision for Open-Ended Writing with Deep Reasoning](https://mlsanity.com/paper/2604.03004/) (2026)
- [IntentRL: Training Proactive User-intent Agents for Open-ended Deep Research via Reinforcement Learning](https://mlsanity.com/paper/2602.03468/) (2026)
- [EcoGym: Evaluating LLMs for Long-Horizon Plan-and-Execute in Interactive Economies](https://mlsanity.com/paper/2602.09514/) (2026)
- [Language Models Don't Know What You Want: Evaluating Personalization in Deep Research Needs Real Users](https://mlsanity.com/paper/2603.16120/) (2026)
- [Towards Knowledgeable Deep Research: Framework and Benchmark](https://mlsanity.com/paper/2604.07720/) (2026)
- [DRACULA: Hunting for the Actions Users Want Deep Research Agents to Execute](https://mlsanity.com/paper/2604.23815/) (2026)
- [Personalized Deep Research Query Refinement with Graph-Scaffolded Evidence Grounding](https://mlsanity.com/paper/2608.05876/) (2026)
- [MobileMem: Learning from a Year of Mobile Experiences](https://mlsanity.com/paper/2608.13606/) (2026)
- [DeepSynth-Eval: Objectively Evaluating Information Consolidation in Deep Survey Writing](https://mlsanity.com/paper/2601.03540/) (2026)
- [PaperFlow: Profiling, Recommending, and Adapting Across Daily Paper Streams](https://mlsanity.com/paper/2606.07454/) (2026)
- [What Happens During Autonomous Deep Research After the User Steps Away?](https://mlsanity.com/paper/2609.33509/) (2026)

## Abstract (arXiv)

Deep Research Agents (DRAs) can autonomously conduct complex investigations and generate comprehensive reports, demonstrating strong real-world potential. However, existing evaluations mostly rely on close-ended benchmarks, while open-ended deep research benchmarks remain scarce and typically neglect personalized scenarios. To bridge this gap, we introduce Personalized Deep Research Bench (PDR-Bench), the first benchmark for evaluating personalization in DRAs. It pairs 50 diverse research tasks across 10 domains with 25 authentic user profiles that combine structured persona attributes with dynamic real-world contexts, yielding 250 realistic user-task queries. To assess system performance, we propose the PQR Evaluation Framework, which jointly measures Personalization Alignment, Content Quality, and Factual Reliability. Our experiments on a range of systems highlight current capabilities and limitations in handling personalized deep research. This work establishes a rigorous foundation for developing and evaluating the next generation of truly personalized AI research assistants.