# Tell Me More! Towards Implicit User Intention Understanding of Language Model Driven Agents

arXiv: https://arxiv.org/abs/2402.09205 · HTML: https://mlsanity.com/paper/2402.09205/

## Key facts

- Cited by 74 papers in the map (data through 14 Oct 2026).
- Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use.
- Builds on 30 of its 73 references that are in the map.
- First posted on arXiv 14 Feb 2024.

## Placed in

- LLM agents and tool use / [General LLM agent frameworks and benchmarks](https://mlsanity.com/agents/general-llm-agent-frameworks-and-benchmarks/)

## Builds on (most cited first, 30 of 30)

- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Toolformer: Language Models Can Teach Themselves to Use Tools](https://mlsanity.com/paper/2302.04761/) (2023)
- [Generative Agents: Interactive Simulacra of Human Behavior](https://mlsanity.com/paper/2304.03442/) (2023)
- [Self-Instruct: Aligning Language Models with Self-Generated Instructions](https://mlsanity.com/paper/2212.10560/) (2022)
- [A Survey on Large Language Model based Autonomous Agents](https://mlsanity.com/paper/2308.11432/) (2023)
- [AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation](https://mlsanity.com/paper/2308.08155/) (2023)
- [ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs](https://mlsanity.com/paper/2307.16789/) (2023)
- [WebGPT: Browser-assisted question-answering with human feedback](https://mlsanity.com/paper/2112.09332/) (2021)
- [The Rise and Potential of Large Language Model Based Agents: A Survey](https://mlsanity.com/paper/2309.07864/) (2023)
- [Gorilla: Large Language Model Connected with Massive APIs](https://mlsanity.com/paper/2305.15334/) (2023)
- [Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agents](https://mlsanity.com/paper/2201.07207/) (2022)
- [Reasoning with Language Model is Planning with World Model](https://mlsanity.com/paper/2305.14992/) (2023)
- [GLM-130B: An Open Bilingual Pre-trained Model](https://mlsanity.com/paper/2210.02414/) (2022)
- [ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debate](https://mlsanity.com/paper/2308.07201/) (2023)
- [Visual ChatGPT: Talking, Drawing and Editing with Visual Foundation Models](https://mlsanity.com/paper/2303.04671/) (2023)
- [Visual Programming: Compositional visual reasoning without training](https://mlsanity.com/paper/2211.11559/) (2022)
- [Large Language Models as Tool Makers](https://mlsanity.com/paper/2305.17126/) (2023)
- [ChatDB: Augmenting LLMs with Databases as Their Symbolic Memory](https://mlsanity.com/paper/2306.03901/) (2023)
- [GeneGPT: Augmenting Large Language Models with Domain Tools for Improved Access to Biomedical Information](https://mlsanity.com/paper/2304.09667/) (2023)
- [Wider and Deeper LLM Networks are Fairer LLM Evaluators](https://mlsanity.com/paper/2308.01862/) (2023)
- [Algorithm of Thoughts: Enhancing Exploration of Ideas in Large Language Models](https://mlsanity.com/paper/2308.10379/) (2023)
- [Analyzing and Characterizing User Intent in Information-seeking Conversations](https://mlsanity.com/paper/1804.08759/) (2018)
- [Learning to Execute Actions or Ask Clarification Questions](https://mlsanity.com/paper/2204.08373/) (2022)
- [Toolink: Linking Toolkit Creation and Using through Chain-of-Solving on Open-Source Model](https://mlsanity.com/paper/2310.05155/) (2023)
- [Human-Centric Research for NLP: Towards a Definition and Guiding Questions](https://mlsanity.com/paper/2207.04447/) (2022)
- [Enhancing Open-Domain Task-Solving Capability of LLMs via Autonomous Tool Integration from GitHub](https://mlsanity.com/paper/2312.17294/) (2023)
- [Zero-Shot Prompting for Implicit Intent Prediction and Recommendation with Commonsense Reasoning](https://mlsanity.com/paper/2210.05901/) (2022)
- [Bayes and Naive Bayes Classifier](https://mlsanity.com/paper/1404.0933/) (2014)

## Built on by (most cited first, 30 of 74)

- [Proactive Agent: Shifting LLM Agents from Reactive Responses to Active Assistance](https://mlsanity.com/paper/2410.12361/) (2024)
- [Aligning to Thousands of Preferences via System Message Generalization](https://mlsanity.com/paper/2405.17977/) (2024)
- [UserBench: An Interactive Gym Environment for User-Centric Agents](https://mlsanity.com/paper/2507.22034/) (2025)
- [Ask-before-Plan: Proactive Language Agents for Real-World Planning](https://mlsanity.com/paper/2406.12639/) (2024)
- [VitaBench: Benchmarking LLM Agents with Versatile Interactive Tasks in Real-world Applications](https://mlsanity.com/paper/2509.26490/) (2025)
- [Re-Invoke: Tool Invocation Rewriting for Zero-Shot Tool Retrieval](https://mlsanity.com/paper/2408.01875/) (2024)
- [SMART: Self-Aware Agent for Tool Overuse Mitigation](https://mlsanity.com/paper/2502.11435/) (2025)
- [ToolOrchestra: Elevating Intelligence via Efficient Model and Tool Orchestration](https://mlsanity.com/paper/2511.21689/) (2025)
- [AGILE: A Novel Reinforcement Learning Framework of LLM Agents](https://mlsanity.com/paper/2405.14751/) (2024)
- [AgentCPM-GUI: Building Mobile-Use Agents with Reinforcement Fine-Tuning](https://mlsanity.com/paper/2506.01391/) (2025)
- [Learning to Ask: When LLM Agents Meet Unclear Instruction](https://mlsanity.com/paper/2409.00557/) (2024)
- [ShortcutsBench: A Large-Scale Real-world Benchmark for API-based Agents](https://mlsanity.com/paper/2407.00132/) (2024)
- [From Language to Action: A Review of Large Language Models as Autonomous Agents and Tool Users](https://mlsanity.com/paper/2508.17281/) (2025)
- [UserRL: Training Interactive User-Centric Agent via Reinforcement Learning](https://mlsanity.com/paper/2509.19736/) (2025)
- [What Should We Engineer in Prompts? Training Humans in Requirement-Driven LLM Use](https://mlsanity.com/paper/2409.08775/) (2024)
- [AgentNoiseBench: Benchmarking Robustness of Tool-Using LLM Agents Under Noisy Condition](https://mlsanity.com/paper/2602.11348/) (2026)
- [CUPID: Evaluating Personalized and Contextualized Alignment of LLMs from Interactions](https://mlsanity.com/paper/2508.01674/) (2025)
- [FingerTip 20K: A Benchmark for Proactive and Personalized Mobile LLM Agents](https://mlsanity.com/paper/2507.21071/) (2025)
- [A Survey on Large Language Models from General Purpose to Medical Applications: Datasets, Methodologies, and Evaluations](https://mlsanity.com/paper/2406.10303/) (2024)
- [When and What to Ask: AskBench and Rubric-Guided RLVR for LLM Clarification](https://mlsanity.com/paper/2602.11199/) (2026)
- [CondAmbigQA: A Benchmark and Dataset for Conditional Ambiguous Question Answering](https://mlsanity.com/paper/2502.01523/) (2025)
- [CompileAgent: Automated Real-World Repo-Level Compilation with Tool-Integrated LLM-based Agent System](https://mlsanity.com/paper/2505.04254/) (2025)
- [AskToAct: Enhancing LLMs Tool Use via Self-Correcting Clarification](https://mlsanity.com/paper/2503.01940/) (2025)
- [AgentMaster: A Multi-Agent Conversational Framework Using A2A and MCP Protocols for Multimodal Information Retrieval and Analysis](https://mlsanity.com/paper/2507.21105/) (2025)
- [ToMAP: Training Opponent-Aware LLM Persuaders with Theory of Mind](https://mlsanity.com/paper/2505.22961/) (2025)
- [InteractComp: Evaluating Search Agents With Ambiguous Queries](https://mlsanity.com/paper/2510.24668/) (2025)
- [InfoPO: Information-Driven Policy Optimization for User-Centric Agents](https://mlsanity.com/paper/2603.00656/) (2026)
- [Do not Abstain! Identify and Solve the Uncertainty](https://mlsanity.com/paper/2506.00780/) (2025)
- [Conversational User-AI Intervention: A Study on Prompt Rewriting for Improved LLM Response Generation](https://mlsanity.com/paper/2503.16789/) (2025)
- [Revisiting Compositional Generalization Capability of Large Language Models Considering Instruction Following Ability](https://mlsanity.com/paper/2506.15629/) (2025)

## Abstract (arXiv)

Current language model-driven agents often lack mechanisms for effective user participation, which is crucial given the vagueness commonly found in user instructions. Although adept at devising strategies and performing tasks, these agents struggle with seeking clarification and grasping precise user intentions. To bridge this gap, we introduce Intention-in-Interaction (IN3), a novel benchmark designed to inspect users' implicit intentions through explicit queries. Next, we propose the incorporation of model experts as the upstream in agent designs to enhance user-agent interaction. Employing IN3, we empirically train Mistral-Interact, a powerful model that proactively assesses task vagueness, inquires user intentions, and refines them into actionable goals before starting downstream agent task execution. Integrating it into the XAgent framework, we comprehensively evaluate the enhanced agent system regarding user instruction understanding and execution, revealing that our approach notably excels at identifying vague user tasks, recovering and summarizing critical missing information, setting precise and necessary agent execution goals, and minimizing redundant tool usage, thus boosting overall efficiency. All the data and codes are released.