Language Models Don't Know What You Want: Evaluating Personalization in Deep Research Needs Real Users
Not on the map. Why?
Abstract (arXiv)
Deep Research (DR) systems help researchers cope with ballooning publishing counts. Such tools synthesize scientific papers to answer research queries, but lack understanding of their users. We address this with MyScholarQA (MySQA), a personalized DR agent that: 1) infers a profile with a user's research interests; 2) proposes personalized actions for a user's input query; and 3) writes a multi-section report for the query that follows user-approved actions. We first test MySQA with NLP's standard protocol: we build a benchmark with synthetic users and LLM judges, where MySQA beats baselines in citation metrics and personalized action-following. However, we suspect this process does not cover all aspects of personalized DR users value, so we interview users in an online version of MySQA to unmask them. We reveal nine nuanced errors of personalized DR undetectable by our LLM judges, and we study qualitative feedback to form lessons for future DR design. In all, we argue for a pillar of personalization that easy-to-use LLM judges can lead NLP to overlook: real progress in personalization is only possible with real users.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 53 of its 108 references that are in the map. First posted on arXiv 17 Mar 2026.
Builds on
53 of 108 references.
References · 53
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
- 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
- 2020Knowledge Distillation: A Surveycited by 1,603
- 2019Evaluating the Factual Consistency of Abstractive Text Summarizationcited by 584
- 2023LaMP: When Large Language Models Meet Personalizationcited by 437
- 2018Generating Wikipedia by Summarizing Long Sequencescited by 335
- 2024Generative Representational Instruction Tuningcited by 233
- 2024BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystackcited by 215
- 2025Know Me, Respond to Me: Benchmarking LLMs for Dynamic User Profiling and Personalized Responses at Scalecited by 178
- 2024RM-Bench: Benchmarking Reward Models of Language Models with Subtlety and Stylecited by 176
38 more
- 2020Deep Learning for Text Style Transfer: A Surveycited by 162
- 2024The PRISM Alignment Dataset: What Participatory, Representative and Individualised Human Feedback Reveals About the Subjective and Multicultural Alignment of Large Language Modelscited by 144
- 2025Deep Research Agents: A Systematic Examination And Roadmapcited by 128
- 2024AutoSurvey: Large Language Models Can Automatically Write Surveyscited by 119
- 2024Democratizing Large Language Models via Personalized Parameter-Efficient Fine-tuningcited by 90
- 2023Human Feedback is not Gold Standardcited by 83
- 2024Modeling Future Conversation Turns to Teach LLMs to Ask Clarifying Questionscited by 76
- 2024OpenScholar: Synthesizing Scientific Literature with Retrieval-augmented LMscited by 72
- 2025Reinforcement Learning from Human Feedbackcited by 58
- 2025AstaBench: Rigorous Benchmarking of AI Agents with a Scientific Research Suitecited by 53
- 2025Which of These Best Describes Multiple Choice Evaluation with LLMs? A) Forced B) Flawed C) Fixable D) All of the Abovecited by 52
- 2024Persona-DB: Efficient Large Language Model Personalization for Response Prediction with Collaborative Data Refinementcited by 49
- 2025SurveyForge: On the Outline Heuristics, Memory-Driven Generation, and Multi-dimensional Evaluation for Automated Survey Writingcited by 49
- 2024LLMs as Research Tools: A Large Scale Survey of Researchers' Usage and Perceptionscited by 47
- 2025Creating General User Models from Computer Usecited by 44
- 2024Into the Unknown Unknowns: Engaged Human Learning through Participation in Language Model Agent Conversationscited by 43
- 2024IdeaSynth: Iterative Research Idea Development Through Evolving and Composing Idea Facets with Literature-Grounded Feedbackcited by 42
- 2024Benchmarks as Microscopes: A Call for Model Metrologycited by 35
- 2025LaMP-QA: A Benchmark for Personalized Long-form Question Answeringcited by 31
- 2024The RealHumanEval: Evaluating Large Language Models' Abilities to Support Programmerscited by 30
- 2025Characterizing Deep Research: A Benchmark and Formal Definitioncited by 30
- 2025SynthesizeMe! Inducing Persona-Guided Prompts for Personalized Reward Models in LLMscited by 29
- 2023Personalized Large Language Model Assistant with Evolving Conditional Memorycited by 25
- 2025Ai2 Scholar QA: Organized Literature Synthesis with Attributioncited by 25
- 2025CUPID: Evaluating Personalized and Contextualized Alignment of LLMs from Interactionscited by 22
- 2022SplitGP: Achieving Both Generalization and Personalization in Federated Learningcited by 20
- 2023Beyond Summarization: Designing AI Support for Real-World Expository Writing Taskscited by 20
- 2025Whose Boat Does it Float? Improving Personalization in Preference Tuning via Inferred User Personascited by 19
- 2023Editable User Profiles for Controllable Text Recommendationcited by 16
- 2025Towards Personalized Deep Research: Benchmarks and Evaluationscited by 16
- 2024Taxonomy Tree Generation from Citation Graphcited by 11
- 2024Step-Back Profiling: Distilling User History for Personalized Scientific Writingcited by 10
- 2024A SMART Mnemonic Sounds like "Glue Tonic": Mixing LLMs with Student Feedback to Make Mnemonic Learning Stickcited by 10
- 2024End-to-end Training for Recommendation with Language-based User Profilescited by 10
- 2025DeepTRACE: Auditing Deep Research AI Systems for Tracking Reliability Across Citations and Evidencecited by 10
- 2023P5: Plug-and-Play Persona Prompting for Personalized Response Selectioncited by 8
- 2024Paper Copilot: A Self-Evolving and Efficient LLM System for Personalized Academic Assistancecited by 8
- 2025A Good Plan is Hard to Find: Aligning Models with Preferences is Misaligned with What Helps Userscited by 6
50 most cited of 53 All 53 (JSON)
Built on by
Citing papers per year: 2026: 2.
2 papers.