mlsanityData to 7 Oct 2026

Exploring the Potential of LLMs as Personalized Assistants: Dataset, Evaluation, and Analysis

arXiv 2506.01262 ↗First posted 2 Jun 2025cs.CLcited by 16

Not on the map. Why?

Abstract (arXiv)

Personalized AI assistants, a hallmark of the human-like capabilities of Large Language Models (LLMs), are a challenging application that intertwines multiple problems in LLM research. Despite the growing interest in the development of personalized assistants, the lack of an open-source conversational dataset tailored for personalization remains a significant obstacle for researchers in the field. To address this research gap, we introduce HiCUPID, a new benchmark to probe and unleash the potential of LLMs to deliver personalized responses. Alongside a conversational dataset, HiCUPID provides a Llama-3.2-based automated evaluation model whose assessment closely mirrors human preferences. We release our dataset, evaluation model, and code at https://github.com/12kimih/HiCUPID.

Key facts. Cited by 16 papers in the map (data through 7 Oct 2026). Builds on 26 of its 49 references that are in the map. First posted on arXiv 2 Jun 2025.

Builds on

26 of 49 references.

References · 26

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2024The Llama 3 Herd of Modelscited by 14,192
  3. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  4. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  5. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  6. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
  7. 2023Qwen Technical Reportcited by 3,273
  8. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,201
  9. 2021Unsupervised Dense Information Retrieval with Contrastive Learningcited by 1,203
  10. 2018Personalizing Dialogue Agents: I have a dog, do you have pets too?cited by 820
  11. 2023LaMP: When Large Language Models Meet Personalizationcited by 425
  12. 2024Scaling Synthetic Data Creation with 1,000,000,000 Personascited by 397
14 more
  1. 2019The Second Conversational Intelligence Challenge (ConvAI2)cited by 178
  2. 2023Uncovering ChatGPT's Capabilities in Recommender Systemscited by 160
  3. 2023Do LLMs Understand User Preferences? Evaluating LLMs On User Rating Predictioncited by 113
  4. 2024Optimization Methods for Personalizing Large Language Models through Retrieval Augmentationcited by 98
  5. 2023Pearl: Personalizing Large Language Model Writing Assistants with Generation-Calibrated Retrieverscited by 91
  6. 2023On the steerability of large language models toward data-driven personascited by 53
  7. 2023Teach LLMs to Personalize -- An Approach inspired by Writing Educationcited by 48
  8. 2020Improving Dialog Systems for Negotiation with Personality Modelingcited by 37
  9. 2020Pchatbot: A Large-Scale Dataset for Personalized Chatbotcited by 26
  10. 2023Enhancing Personalized Dialogue Generation with Contrastive Latent Variables: Combining Sparse and Dense Personacited by 24
  11. 2023Large Language Models for User Interest Journeyscited by 24
  12. 2023Editing Personality for Large Language Modelscited by 22
  13. 2020Refine and Imitate: Reducing Repetition and Inconsistency in Persuasion Dialogues via Reinforcement Learning and Human Demonstrationcited by 20
  14. 2024Evaluating the Performance of Large Language Models via Debatescited by 19

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 3; 2026: 13.

16 papers.

Sort:

By area:

  1. 2026ASTRA-bench: Evaluating Tool-Use Agent Reasoning and Action Planning with Personal User Contextcited by 14
  2. 2026Your Language Model Secretly Contains Personality Subnetworkscited by 4
  3. 2025AgentCoMa: A Compositional Benchmark Mixing Commonsense and Mathematical Reasoning in Real-World Scenarioscited by 3
  4. 2026Responsible Agentic AI Requires Explicit Provenancecited by 3
  5. 2026HingeMem: Boundary Guided Long-Term Memory with Query Adaptive Retrieval for Scalable Dialoguescited by 2
  6. 2025PREFINE: Personalized Story Generation via Simulated User Critics and User-Specific Rubric Generationcited by 1
  7. 2025LikeBench: Evaluating Subjective Likability in LLMs for Personalizationcited by 1
  8. 2026Contextualized Visual Personalization in Vision-Language Modelscited by 1
  9. 2026What to Remember, What to Reveal: Privacy-Aware Memory for Conversational Agentscited by 1
  10. 2026Hypotheses-Guided Self Distillation for Continual Personalizationcited by 1
  11. 2026LUNAR: Benchmarking Personalized Large Language Models on UNiversal User BehAvioR Logscited by 0
  12. 2026SPIEval: Evaluating Large Language Models as Mobile Assistants over Scattered Personal Informationcited by 0
4 more
  1. 2026VIBE-Bench: Evaluating Personalized Large Language Models When Profiles Don't Mean Preferencescited by 0
  2. 2026PRAGMA: Evaluating Personalized Guidance with Memory Alignment in Lifelong Conversationscited by 0
  3. 2026One Prompt Does Not Fit All: Self-Meta-Evolve for Personalized Information Extractioncited by 0
  4. 2026Unlocking Latent Personalization in LLMscited by 0