# Is Passive Expertise-Based Personalization Enough? A Case Study in AI-Assisted Test-Taking

arXiv: https://arxiv.org/abs/2511.23376 · HTML: https://mlsanity.com/paper/2511.23376/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 48 references that are in the map.
- First posted on arXiv 28 Nov 2025.

## Builds on (most cited first, 18 of 18)

- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [LaMP: When Large Language Models Meet Personalization](https://mlsanity.com/paper/2304.11406/) (2023)
- [Optimizing Instructions and Demonstrations for Multi-Stage Language Model Programs](https://mlsanity.com/paper/2406.11695/) (2024)
- [Hello Again! LLM-powered Personalized Agent for Long-term Dialogue](https://mlsanity.com/paper/2406.05925/) (2024)
- [Conversational Health Agents: A Personalized LLM-Powered Agent Framework](https://mlsanity.com/paper/2310.02374/) (2023)
- [Personalized Dialogue Generation with Diversified Traits](https://mlsanity.com/paper/1901.09672/) (2019)
- [Exploring Knowledge Tracing in Tutor-Student Dialogues using LLMs](https://mlsanity.com/paper/2409.16490/) (2024)
- [Learning to Rewrite Prompts for Personalized Text Generation](https://mlsanity.com/paper/2310.00152/) (2023)
- [Advancing Human-AI Complementarity: The Impact of User Expertise and Algorithmic Tuning on Joint Decision Making](https://mlsanity.com/paper/2208.07960/) (2022)
- [Personalization in Goal-Oriented Dialog](https://mlsanity.com/paper/1706.07503/) (2017)
- [Doing Personal LAPS: LLM-Augmented Dialogue Construction for Personalized Multi-Session Conversational Search](https://mlsanity.com/paper/2405.03480/) (2024)
- [Learning Agent-based Modeling with LLM Companions: Experiences of Novices and Experts Using ChatGPT & NetLogo Chat](https://mlsanity.com/paper/2401.17163/) (2024)
- [Towards Modeling Learner Performance with Large Language Models](https://mlsanity.com/paper/2403.14661/) (2024)
- [Learning Personalized End-to-End Goal-Oriented Dialog](https://mlsanity.com/paper/1811.04604/) (2018)
- [CLST: Cold-Start Mitigation in Knowledge Tracing by Aligning a Generative Language Model as a Students' Knowledge Tracer](https://mlsanity.com/paper/2406.10296/) (2024)
- [ELI-Why: Evaluating the Pedagogical Utility of Language Model Explanations](https://mlsanity.com/paper/2506.14200/) (2025)
- [ProfiLLM: An LLM-Based Framework for Implicit Profiling of Chatbot Users](https://mlsanity.com/paper/2506.13980/) (2025)
- [The Impact of Technical Domain Expertise on Search Behavior and Task Outcome](https://mlsanity.com/paper/1512.07051/) (2015)

## Built on by (most cited first, 1 of 1)

- [Beyond Direct Identifiers: Probabilistic Privacy Risk Estimation for Privacy-Conscious LLM Query Delegation](https://mlsanity.com/paper/2608.09140/) (2026)

## Abstract (arXiv)

Novice and expert users have different systematic preferences in task-oriented dialogues. However, whether catering to these preferences actually improves user experience and task performance remains understudied. To investigate the effects of expertise-based personalization, we first built a version of an enterprise AI assistant with passive personalization. We then conducted a user study where participants completed timed exams, aided by the two versions of the AI assistant. Preliminary results indicate that passive personalization helps reduce task load and improve assistant perception, but reveal task-specific limitations that can be addressed through providing more user agency. These findings underscore the importance of combining active and passive personalization to optimize user experience and effectiveness in enterprise task-oriented environments.