# Clarify User Expertise: Towards Proactive Conversational Agents Tailoring Responses to User Proficiency

arXiv: https://arxiv.org/abs/2608.22266 · HTML: https://mlsanity.com/paper/2608.22266/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use.
- Builds on 23 of its 58 references that are in the map.
- First posted on arXiv 23 Aug 2026.

## Placed in

- LLM agents and tool use / [General LLM agent frameworks and benchmarks](https://mlsanity.com/agents/general-llm-agent-frameworks-and-benchmarks/)

## Builds on (most cited first, 23 of 23)

- [Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge](https://mlsanity.com/paper/1803.05457/) (2018)
- [MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmark](https://mlsanity.com/paper/2406.01574/) (2024)
- [ExpertPrompting: Instructing Large Language Models to be Distinguished Experts](https://mlsanity.com/paper/2305.14688/) (2023)
- [Multi-hop Reading Comprehension through Question Decomposition and Rescoring](https://mlsanity.com/paper/1906.02916/) (2019)
- [Prompting and Evaluating Large Language Models for Proactive Dialogues: Clarification, Target-guided, and Non-collaboration](https://mlsanity.com/paper/2305.13626/) (2023)
- [Persona-DB: Efficient Large Language Model Personalization for Response Prediction with Collaborative Data Refinement](https://mlsanity.com/paper/2402.11060/) (2024)
- [Towards Human-centered Proactive Conversational Agents](https://mlsanity.com/paper/2404.12670/) (2024)
- [INSCIT: Information-Seeking Conversations with Mixed-Initiative Interactions](https://mlsanity.com/paper/2207.00746/) (2022)
- [Analyzing and Learning from User Interactions for Search Clarification](https://mlsanity.com/paper/2006.00166/) (2020)
- [Learning Agent-based Modeling with LLM Companions: Experiences of Novices and Experts Using ChatGPT & NetLogo Chat](https://mlsanity.com/paper/2401.17163/) (2024)
- [Strength Lies in Differences! Improving Strategy Planning for Non-collaborative Dialogues via Diversified User Simulation](https://mlsanity.com/paper/2403.06769/) (2024)
- [Multiple Choice Questions based Multi-Interest Policy Learning for Conversational Recommendation](https://mlsanity.com/paper/2112.11775/) (2021)
- [README: Bridging Medical Jargon and Lay Understanding for Patient Education through Data-Centric NLP](https://mlsanity.com/paper/2312.15561/) (2023)
- [STYLE: Improving Domain Transferability of Asking Clarification Questions in Large Language Model Powered Conversational Agents](https://mlsanity.com/paper/2405.12059/) (2024)
- [PRINCIPLES: Synthetic Strategy Memory for Proactive Dialogue Agents](https://mlsanity.com/paper/2509.17459/) (2025)
- ["In Dialogues We Learn": Towards Personalized Dialogue Without Pre-defined Profiles through In-Dialogue Learning](https://mlsanity.com/paper/2403.03102/) (2024)
- [DYSTIL: Dynamic Strategy Induction with Large Language Models for Reinforcement Learning](https://mlsanity.com/paper/2505.03209/) (2025)
- [ARAIDA: Analogical Reasoning-Augmented Interactive Data Annotation](https://mlsanity.com/paper/2405.11912/) (2024)
- [LLMTaxo: Leveraging Large Language Models for Constructing Taxonomy of Factual Claims from Social Media](https://mlsanity.com/paper/2504.12325/) (2025)
- [CAVE: Detecting and Explaining Commonsense Anomalies in Visual Environments](https://mlsanity.com/paper/2510.26006/) (2025)
- [Breaking the Stigma! Unobtrusively Probe Symptoms in Depression Disorder Diagnosis Dialogue](https://mlsanity.com/paper/2501.15260/) (2025)
- [Towards Proactive Information Probing: Customer Service Chatbots Harvesting Value from Conversation](https://mlsanity.com/paper/2604.11077/) (2026)
- [Clearing the Fog: Towards Installing and Refining Proactive Exploration Capabilities in LLM Agents](https://mlsanity.com/paper/2608.14339/) (2026)

## Abstract (arXiv)

In the context of information seeking, conversational agents are undergoing an evolution from reactive tools to proactive, personalized assistants. A critical aspect of this evolution is the ability to tailor strategic interactions to a user's unique needs and expectations. Unlike existing studies that focus on proactively clarifying query ambiguities, we center on clarifying the user's expertise in order to tailor responses for better user comprehension. We find that existing agents struggle to determine user expertise from queries alone, a limitation that prevents them from dynamically adapting their responses. To address this gap, we introduce PASSING to empower the agent to proactively clarify a user's expertise through targeted inquiries. This is achieved by our What-to-ask and How-to-ask strategies, induced by LLM self-play. Our extensive experiments also show our superiority. We believe that PASSING represents a crucial step towards creating more human-centric conversational agents.