# IPQA: A Benchmark for Core Intent Identification in Personalized Question Answering

arXiv: https://arxiv.org/abs/2510.23536 · HTML: https://mlsanity.com/paper/2510.23536/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 42 references that are in the map.
- First posted on arXiv 27 Oct 2025.

## Builds on (most cited first, 25 of 25)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment](https://mlsanity.com/paper/2303.16634/) (2023)
- [Gemma 3 Technical Report](https://mlsanity.com/paper/2503.19786/) (2025)
- [A Call for Clarity in Reporting BLEU Scores](https://mlsanity.com/paper/1804.08771/) (2018)
- [Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Models](https://mlsanity.com/paper/2506.05176/) (2025)
- [Qwen2.5-Coder Technical Report](https://mlsanity.com/paper/2409.12186/) (2024)
- [LaMP: When Large Language Models Meet Personalization](https://mlsanity.com/paper/2304.11406/) (2023)
- [Efficient Intent Detection with Dual Sentence Encoders](https://mlsanity.com/paper/2003.04807/) (2020)
- [An Evaluation Dataset for Intent Classification and Out-of-Scope Prediction](https://mlsanity.com/paper/1909.02027/) (2019)
- [Snips Voice Platform: an embedded Spoken Language Understanding system for private-by-design voice interfaces](https://mlsanity.com/paper/1805.10190/) (2018)
- [LongLaMP: A Benchmark for Personalized Long-form Text Generation](https://mlsanity.com/paper/2407.11016/) (2024)
- [Deep Open Intent Classification with Adaptive Decision Boundary](https://mlsanity.com/paper/2012.10209/) (2020)
- [Analyzing Dataset Annotation Quality Management in the Wild](https://mlsanity.com/paper/2307.08153/) (2023)
- [LaMP-QA: A Benchmark for Personalized Long-form Question Answering](https://mlsanity.com/paper/2506.00137/) (2025)
- [ExPerT: Effective and Explainable Evaluation of Personalized Long-Form Text Generation](https://mlsanity.com/paper/2501.14956/) (2025)
- [TEXTOIR: An Integrated and Visualized Platform for Text Open Intent Recognition](https://mlsanity.com/paper/2110.15063/) (2021)
- [Review-driven Personalized Preference Reasoning with Large Language Models for Recommendation](https://mlsanity.com/paper/2408.06276/) (2024)
- [Learning Discriminative Representations and Decision Boundaries for Open Intent Detection](https://mlsanity.com/paper/2203.05823/) (2022)
- [UQABench: Evaluating User Embedding for Prompting LLMs in Personalized Question Answering](https://mlsanity.com/paper/2502.19178/) (2025)
- [SE-PQA: Personalized Community Question Answering](https://mlsanity.com/paper/2306.16261/) (2023)
- [MT-RAIG: Novel Benchmark and Evaluation Framework for Retrieval-Augmented Insight Generation over Multiple Tables](https://mlsanity.com/paper/2502.11735/) (2025)
- [BlendX: Complex Multi-Intent Detection with Blended Patterns](https://mlsanity.com/paper/2403.18277/) (2024)
- [Can Large Language Models be Effective Online Opinion Miners?](https://mlsanity.com/paper/2505.15695/) (2025)

## Built on by (most cited first, 2 of 2)

- [Learning to Reason for Multi-Step Retrieval of Personal Context in Personalized Question Answering](https://mlsanity.com/paper/2602.19317/) (2026)
- [Training LLMs with Reinforcement Learning for Intent-Aware Personalized Question Answering](https://mlsanity.com/paper/2605.12645/) (2026)

## Abstract (arXiv)

Intent identification serves as the foundation for generating appropriate responses in personalized question answering (PQA). However, existing benchmarks evaluate only response quality or retrieval performance without directly measuring intent identification capabilities. This gap is critical because without understanding which intents users prioritize, systems cannot generate responses satisfying individual information needs. To address this, we introduce the concept of core intents: intents users prioritize when selecting answers to satisfy their information needs. To evaluate these core intents, we propose IPQA, a benchmark for core Intent identification in Personalized Question Answering. Since users do not explicitly state their prioritized intents, we derive core intents from observable behavior patterns in answer selection, grounded in satisficing theory where users choose answers meeting their acceptance thresholds. We construct a dataset with various domains through systematic filtering, LLM-based annotation, and rigorous quality control combining automated verification with human validation. Experimental evaluations across state-of-the-art language models reveal that current systems struggle with core intent identification in personalized contexts. Models fail to identify core intents from user histories, with performance degrading as question complexity increases. The code and dataset will be made publicly available to facilitate future research in this direction.