mlsanityData to 14 Oct 2026

LaMP-QA: A Benchmark for Personalized Long-form Question Answering

arXiv 2506.00137 ↗First posted 30 May 2025cs.CL cs.IR cs.LGcited by 31

Not on the map. Why?

Abstract (arXiv)

Personalization is essential for question answering systems that are user-centric. Despite its importance, personalization in answer generation has been relatively underexplored. This is mainly due to lack of resources for training and evaluating personalized question answering systems. We address this gap by introducing LaMP-QA -- a benchmark designed for evaluating personalized long-form answer generation. The benchmark covers questions from three major categories: (1) Arts & Entertainment, (2) Lifestyle & Personal Development, and (3) Society & Culture, encompassing over 45 subcategories in total. To assess the quality and potential impact of the LaMP-QA benchmark for personalized question answering, we conduct comprehensive human and automatic evaluations, to compare multiple evaluation strategies for evaluating generated personalized responses and measure their alignment with human preferences. Furthermore, we benchmark a number of non-personalized and personalized approaches based on open-source and proprietary large language models. Our results show that incorporating the personalized context provided leads to up to 39% performance improvements. The benchmark is publicly released to support future research in this area.

Key facts. Cited by 31 papers in the map (data through 14 Oct 2026). Builds on 26 of its 45 references that are in the map. First posted on arXiv 30 May 2025.

Builds on

26 of 45 references.

References · 26

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
  3. 2014Sequence to Sequence Learning with Neural Networkscited by 2,986
  4. 2019The Curious Case of Neural Text Degenerationcited by 2,916
  5. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,276
  6. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,998
  7. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,605
  8. 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,282
  9. 2021Unsupervised Dense Information Retrieval with Contrastive Learningcited by 1,216
  10. 2023LaMP: When Large Language Models Meet Personalizationcited by 437
  11. 2023Large Language Models Are State-of-the-Art Evaluators of Translation Qualitycited by 420
  12. 2019Deep Learning Recommendation Model for Personalization and Recommendation Systemscited by 327
14 more
  1. 2023Personalized Soups: Personalized Large Language Model Alignment via Post-hoc Parameter Mergingcited by 280
  2. 2024Optimization Methods for Personalizing Large Language Models through Retrieval Augmentationcited by 99
  3. 2024LongLaMP: A Benchmark for Personalized Long-form Text Generationcited by 97
  4. 2023Pearl: Personalizing Large Language Model Writing Assistants with Generation-Calibrated Retrieverscited by 92
  5. 2024Personalized Pieces: Efficient Personalized Large Language Models through Collaborative Effortscited by 74
  6. 2023Teach LLMs to Personalize -- An Approach inspired by Writing Educationcited by 48
  7. 2023LLM-based Medical Assistant Personalization with Short- and Long-Term Memory Coordinationcited by 46
  8. 2025Personalized Generation In Large Model Era: A Surveycited by 45
  9. 2023Learning to Rewrite Prompts for Personalized Text Generationcited by 40
  10. 2025Reasoning-Enhanced Self-Training for Long-Form Personalized Text Generationcited by 27
  11. 2025ExPerT: Effective and Explainable Evaluation of Personalized Long-Form Text Generationcited by 26
  12. 2024Overview of the TREC 2023 NeuCLIR Trackcited by 17
  13. 2025Beyond Factual Accuracy: Evaluating Coverage of Diverse Factual Information in Long-form Text Generationcited by 15
  14. 2024Corporate Communication Companion (CCC): An LLM-empowered Writing Assistant for Workplace Social Mediacited by 14

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 5; 2026: 26.

31 papers.

Sort:

By area:

  1. 2025Think-While-Generating: On-the-Fly Reasoning for Personalized Long-Form Generationcited by 30
  2. 2025Instant Personalized Large Language Model Adaptation via Hypernetworkcited by 18
  3. 2026Toward Personalized LLM-Powered Agents: Foundations, Evaluation, and Future Directionscited by 14
  4. 2025PrivacyBench: A Conversational Benchmark for Evaluating Privacy in Personalized AIcited by 12
  5. 2026InfoPO: Information-Driven Policy Optimization for User-Centric Agentscited by 12
  6. 2026AlpsBench: An LLM Personalization Benchmark for Real-Dialogue Memorization and Preference Alignmentcited by 10
  7. 2026Personalized RewardBench: Evaluating Reward Models with Human Aligned Personalizationcited by 7
  8. 2025Learning from Natural Language Feedback for Personalized Question Answeringcited by 5
  9. 2026P-GenRM: Personalized Generative Reward Model with Test-time User-based Scalingcited by 4
  10. 2026Rethinking Personalization in Large Language Models at the Token Levelcited by 3
  11. 2026Personal Visual Context Learning in Large Multimodal Modelscited by 3
  12. 2025IPQA: A Benchmark for Core Intent Identification in Personalized Question Answeringcited by 2
19 more
  1. 2026Language Models Don't Know What You Want: Evaluating Personalization in Deep Research Needs Real Userscited by 2
  2. 2026Training LLMs with Reinforcement Learning for Intent-Aware Personalized Question Answeringcited by 2
  3. 2026Preference-Aware Rubric Learning for Personalized Evaluationcited by 2
  4. 2026LUNAR: Benchmarking Personalized Large Language Models on UNiversal User BehAvioR Logscited by 2
  5. 2026PRISP: Privacy-Safe Few-Shot Personalization via Lightweight Adaptationcited by 1
  6. 2026Improving User Privacy in Personalized Generation: Client-Side Retrieval-Augmented Modification of Server-Side Generated Speculationscited by 1
  7. 2026CoPA: Benchmarking Personalized Question Answering with Data-Informed Cognitive Factorscited by 1
  8. 2026Learning to Adapt Cross-Domain Preferences via Meta-LoRA for LLM Personalizationcited by 1
  9. 2026Can LLMs Take Retrieved Information with a Grain of Salt?cited by 0
  10. 2026An Interactive Paradigm for Deep Researchcited by 0
  11. 2026Evaluating LLM Personalization via Semantic Constraint Verificationcited by 0
  12. 2026PersonaMem-v3: Toward Omni-Platform Personal Intelligence for Holistic User Understanding, Recommendation, and Agentic Taskscited by 0
  13. 2026Evaluating the Hidden Costs of Personalization in Large Language Modelscited by 0
  14. 2026Auditing and Mitigating Privacy Leakage in Cloud-Edge Collaborative Decodingcited by 0
  15. 2026VIBE-Bench: Evaluating Personalized Large Language Models When Profiles Don't Mean Preferencescited by 0
  16. 2026One Prompt Does Not Fit All: Self-Meta-Evolve for Personalized Information Extractioncited by 0
  17. 2026Rubric-Aware On-Policy Self-Distillation for LLM Personalizationcited by 0
  18. 2026MemFold: Learning Compact Soft Memory for Long-Context Personalization via On-Policy Optimizationcited by 0
  19. 2026PersonaFactor: Preference Factors with Dynamic Selection for LLM Personalizationcited by 0