# UQABench: Evaluating User Embedding for Prompting LLMs in Personalized Question Answering

arXiv: https://arxiv.org/abs/2502.19178 · HTML: https://mlsanity.com/paper/2502.19178/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Feb 2025.

## Built on by (most cited first, 10 of 10)

- [COMI: Coarse-to-fine Context Compression via Marginal Information Gain](https://mlsanity.com/paper/2602.01719/) (2026)
- [Read As Human: Compressing Context via Parallelizable Close Reading and Skimming](https://mlsanity.com/paper/2602.01840/) (2026)
- [ECKGBench: Benchmarking Large Language Models in E-commerce Leveraging Knowledge Graph](https://mlsanity.com/paper/2503.15990/) (2025)
- [IPQA: A Benchmark for Core Intent Identification in Personalized Question Answering](https://mlsanity.com/paper/2510.23536/) (2025)
- [CoPA: Benchmarking Personalized Question Answering with Data-Informed Cognitive Factors](https://mlsanity.com/paper/2604.14773/) (2026)
- [POLAR:A Per-User Association Test in Embedding Space](https://mlsanity.com/paper/2603.15950/) (2026)
- [Bridging Textual Profiles and Latent User Embeddings for Personalization](https://mlsanity.com/paper/2605.06981/) (2026)
- [Fighting Numerical Hallucinations via Data-centric Compilation for Online Financial QA](https://mlsanity.com/paper/2605.31064/) (2026)
- [Substrate Asymmetry in User-Side Memory: A Diagnostic Framework](https://mlsanity.com/paper/2606.11712/) (2026)
- [CUEing User Simulators: Calibrated User Embeddings for Multi-Turn Benchmarking](https://mlsanity.com/paper/2610.02460/) (2026)

## Abstract (arXiv)

Large language models (LLMs) achieve remarkable success in natural language processing (NLP). In practical scenarios like recommendations, as users increasingly seek personalized experiences, it becomes crucial to incorporate user interaction history into the context of LLMs to enhance personalization. However, from a practical utility perspective, user interactions' extensive length and noise present challenges when used directly as text prompts. A promising solution is to compress and distill interactions into compact embeddings, serving as soft prompts to assist LLMs in generating personalized responses. Although this approach brings efficiency, a critical concern emerges: Can user embeddings adequately capture valuable information and prompt LLMs? To address this concern, we propose name, a benchmark designed to evaluate the effectiveness of user embeddings in prompting LLMs for personalization. We establish a fair and standardized evaluation process, encompassing pre-training, fine-tuning, and evaluation stages. To thoroughly evaluate user embeddings, we design three dimensions of tasks: sequence understanding, action prediction, and interest perception. These evaluation tasks cover the industry's demands in traditional recommendation tasks, such as improving prediction accuracy, and its aspirations for LLM-based methods, such as accurately understanding user interests and enhancing the user experience. We conduct extensive experiments on various state-of-the-art methods for modeling user embeddings. Additionally, we reveal the scaling laws of leveraging user embeddings to prompt LLMs. The benchmark is available online.