# Active Task Disambiguation with LLMs

arXiv: https://arxiv.org/abs/2502.04485 · HTML: https://mlsanity.com/paper/2502.04485/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use.
- First posted on arXiv 6 Feb 2025.

## Placed in

- LLM agents and tool use / [General LLM agent frameworks and benchmarks](https://mlsanity.com/agents/general-llm-agent-frameworks-and-benchmarks/)

## Built on by (most cited first, 23 of 23)

- [BED-LLM: Intelligent Information Gathering with LLMs and Bayesian Experimental Design](https://mlsanity.com/paper/2508.21184/) (2025)
- [Agentic Abstention: Do Agents Know When to Stop Instead of Act?](https://mlsanity.com/paper/2606.28733/) (2026)
- [ToMAP: Training Opponent-Aware LLM Persuaders with Theory of Mind](https://mlsanity.com/paper/2505.22961/) (2025)
- [When Two LLMs Debate, Both Think They'll Win](https://mlsanity.com/paper/2505.19184/) (2025)
- [Offline RL by Reward-Weighted Fine-Tuning for Conversation Optimization](https://mlsanity.com/paper/2506.06964/) (2025)
- [LPS-Bench: Benchmarking Safety Awareness of Computer-Use Agents in Long-Horizon Planning under Benign and Adversarial Scenarios](https://mlsanity.com/paper/2602.03255/) (2026)
- [SafeCommit: Certifying When Memory-Grounded Agents May Safely Act](https://mlsanity.com/paper/2608.04289/) (2026)
- [LLMs are not (consistently) Bayesian: Quantifying internal (in)consistencies of LLMs' probabilistic beliefs](https://mlsanity.com/paper/2605.06915/) (2026)
- [Ask Early, Ask Late, Ask Right: When Does Clarification Timing Matter for Long-Horizon Agents?](https://mlsanity.com/paper/2605.07937/) (2026)
- [The Perceptual Bandwidth Bottleneck in Vision-Language Models: Active Visual Reasoning via Sequential Experimental Design](https://mlsanity.com/paper/2605.01345/) (2026)
- [Amortising Bayesian Experimental Design for Sequential Information Gathering in LLMs](https://mlsanity.com/paper/2607.03426/) (2026)
- [ARMove: Learning to Predict Human Mobility through Agentic Reasoning](https://mlsanity.com/paper/2604.17419/) (2026)
- [HAS-Bench: Evaluating LLM-Based Human-Agent Systems under Configurable Human Participation](https://mlsanity.com/paper/2607.04329/) (2026)
- [Adaptive Prompt Elicitation for Text-to-Image Generation](https://mlsanity.com/paper/2602.04713/) (2026)
- [Interactive Clarification for Cloud Infrastructure-as-Code Synthesis](https://mlsanity.com/paper/2604.02382/) (2026)
- [Uncertainty as a Planning Signal: Multi-Turn Decision Making for Goal-Oriented Conversation](https://mlsanity.com/paper/2604.03924/) (2026)
- [Alignment has a Fantasia Problem](https://mlsanity.com/paper/2604.21827/) (2026)
- [Adaptive Querying with AI Persona Priors](https://mlsanity.com/paper/2605.00696/) (2026)
- [When to Ask a Question: Understanding Communication Strategies in Generative AI Tools](https://mlsanity.com/paper/2605.11240/) (2026)
- [AFFORDANCE20Q: Evaluating Affordance Reasoning from Physical Properties](https://mlsanity.com/paper/2606.14240/) (2026)
- [NL-PAC: Specification Ambiguity and Certified Minimax Risk Floors in LLM-Mediated Supervision](https://mlsanity.com/paper/2607.08961/) (2026)
- [Active Inference as Context Acquisition for AI Agents](https://mlsanity.com/paper/2608.19202/) (2026)
- [Clarify the User or Verify the World? Uncertainty Routing for Proactive Agents](https://mlsanity.com/paper/2609.32255/) (2026)

## Abstract (arXiv)

Despite the impressive performance of large language models (LLMs) across various benchmarks, their ability to address ambiguously specified problems--frequent in real-world interactions--remains underexplored. To address this gap, we introduce a formal definition of task ambiguity and frame the problem of task disambiguation through the lens of Bayesian Experimental Design. By posing clarifying questions, LLM agents can acquire additional task specifications, progressively narrowing the space of viable solutions and reducing the risk of generating unsatisfactory outputs. Yet, generating effective clarifying questions requires LLM agents to engage in a form of meta-cognitive reasoning, an ability LLMs may presently lack. Our proposed approach of active task disambiguation enables LLM agents to generate targeted questions maximizing the information gain. Effectively, this approach shifts the load from implicit to explicit reasoning about the space of viable solutions. Empirical results demonstrate that this form of question selection leads to more effective task disambiguation in comparison to approaches relying on reasoning solely within the space of questions.