# The Personalization Paradox: Semantic Loss vs. Reasoning Gains in Agentic AI Q&A

arXiv: https://arxiv.org/abs/2512.04343 · HTML: https://mlsanity.com/paper/2512.04343/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning.
- Builds on 9 of its 33 references that are in the map.
- First posted on arXiv 4 Dec 2025.

## Placed in

- LLM reasoning / [Reinforcement learning for reasoning](https://mlsanity.com/lr/reinforcement-learning-for-reasoning/)
- Retrieval-augmented generation / [Agentic RAG, agents and memory](https://mlsanity.com/rag/agentic-rag-agents-and-memory/)
- LLM agents and tool use / [Agent memory systems](https://mlsanity.com/agents/agent-memory-systems/)

## Builds on (most cited first, 9 of 9)

- [Retrieval-Augmented Generation for Large Language Models: A Survey](https://mlsanity.com/paper/2312.10997/) (2023)
- [A Survey of Personalized Large Language Models: Progress and Future Directions](https://mlsanity.com/paper/2502.11528/) (2025)
- [A Systematic Literature Review of User Trust in AI-Enabled Systems: An HCI Perspective](https://mlsanity.com/paper/2304.08795/) (2023)
- [Few-shot Personalization of LLMs with Mis-aligned Responses](https://mlsanity.com/paper/2406.18678/) (2024)
- [Personalizing Task-oriented Dialog Systems via Zero-shot Generalizable Reward Function](https://mlsanity.com/paper/2303.13797/) (2023)
- [Unimib Assistant: designing a student-friendly RAG-based chatbot for all their needs](https://mlsanity.com/paper/2411.19554/) (2024)
- [Applying Fine-Tuned LLMs for Reducing Data Needs in Load Profile Analysis](https://mlsanity.com/paper/2406.02479/) (2024)
- [A Survey of Affective Recommender Systems: Modeling Attitudes, Emotions, and Moods for Personalization](https://mlsanity.com/paper/2508.20289/) (2025)
- ["Do it my way!": Impact of Customizations on Trust perceptions in Human-Robot Collaboration](https://mlsanity.com/paper/2310.18791/) (2023)

## Abstract (arXiv)

AIVisor, an agentic retrieval-augmented LLM for student advising, was used to examine how personalization affects system performance across multiple evaluation dimensions. Using twelve authentic advising questions intentionally designed to stress lexical precision, we compared ten personalized and non-personalized system configurations and analyzed outcomes with a Linear Mixed-Effects Model across lexical (BLEU, ROUGE-L), semantic (METEOR, BERTScore), and grounding (RAGAS) metrics. Results showed a consistent trade-off: personalization reliably improved reasoning quality and grounding, yet introduced a significant negative interaction on semantic similarity, driven not by poorer answers but by the limits of current metrics, which penalize meaningful personalized deviations from generic reference texts. This reveals a structural flaw in prevailing LLM evaluation methods, which are ill-suited for assessing user-specific responses. The fully integrated personalized configuration produced the highest overall gains, suggesting that personalization can enhance system effectiveness when evaluated with appropriate multidimensional metrics. Overall, the study demonstrates that personalization produces metric-dependent shifts rather than uniform improvements and provides a methodological foundation for more transparent and robust personalization in agentic AI.