# Actively Resolving Contextual Uncertainty for Underspecified Tasks in Natural Language

arXiv: https://arxiv.org/abs/2609.30428 · HTML: https://mlsanity.com/paper/2609.30428/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 40 references that are in the map.
- First posted on arXiv 24 Sep 2026.

## Builds on (most cited first, 30 of 32)

- [RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control](https://mlsanity.com/paper/2307.15818/) (2023)
- [π0.5: a Vision-Language-Action Model with Open-World Generalization](https://mlsanity.com/paper/2504.16054/) (2025)
- [Robots That Ask For Help: Uncertainty Alignment for Large Language Model Planners](https://mlsanity.com/paper/2307.01928/) (2023)
- [VLFM: Vision-Language Frontier Maps for Zero-Shot Semantic Navigation](https://mlsanity.com/paper/2312.03275/) (2023)
- [RoCo: Dialectic Multi-Robot Collaboration with Large Language Models](https://mlsanity.com/paper/2307.04738/) (2023)
- [Open-vocabulary Queryable Scene Representations for Real World Planning](https://mlsanity.com/paper/2209.09874/) (2022)
- [SMART-LLM: Smart Multi-Agent Robot Task Planning using Large Language Models](https://mlsanity.com/paper/2309.10062/) (2023)
- [Scalable Multi-Robot Collaboration with Large Language Models: Centralized or Decentralized Systems?](https://mlsanity.com/paper/2309.15943/) (2023)
- [Explore until Confident: Efficient Exploration for Embodied Question Answering](https://mlsanity.com/paper/2403.15941/) (2024)
- [The Marathon 2: A Navigation System](https://mlsanity.com/paper/2003.00368/) (2020)
- [LLaRA: Supercharging Robot Learning Data for Vision-Language Policy](https://mlsanity.com/paper/2406.20095/) (2024)
- [FUEL: Fast UAV Exploration using Incremental Frontier Structure and Hierarchical Planning](https://mlsanity.com/paper/2010.11561/) (2020)
- [Optimal Scene Graph Planning with Large Language Model Guidance](https://mlsanity.com/paper/2309.09182/) (2023)
- [Interactive Planning Using Large Language Models for Partially Observable Robotics Tasks](https://mlsanity.com/paper/2312.06876/) (2023)
- [Introspective Planning: Aligning Robots' Uncertainty with Inherent Task Ambiguity](https://mlsanity.com/paper/2402.06529/) (2024)
- [RayFronts: Open-Set Semantic Ray Frontiers for Online Scene Understanding and Exploration](https://mlsanity.com/paper/2504.06994/) (2025)
- [Describe Anything Anywhere At Any Moment](https://mlsanity.com/paper/2512.00565/) (2025)
- [SPINE: Online Semantic Planning for Missions with Incomplete Natural Language Specifications in Unstructured Environments](https://mlsanity.com/paper/2410.03035/) (2024)
- [RT-GuIDE: Real-Time Gaussian Splatting for Information-Driven Exploration](https://mlsanity.com/paper/2409.18122/) (2024)
- [LLM-Guided Probabilistic Program Induction for POMDP Model Estimation](https://mlsanity.com/paper/2505.02216/) (2025)
- [Distilling On-device Language Models for Robot Planning with Minimal Human Intervention](https://mlsanity.com/paper/2506.17486/) (2025)
- [Hierarchical LLMs In-the-Loop Optimization for Real-Time Multi-Robot Target Tracking under Unknown Hazards](https://mlsanity.com/paper/2409.12274/) (2024)
- [Can Large Language Models Solve Robot Routing?](https://mlsanity.com/paper/2403.10795/) (2024)
- [RADSeg: Unleashing Parameter and Compute Efficient Zero-Shot Open-Vocabulary Segmentation Using Agglomerative Models](https://mlsanity.com/paper/2511.19704/) (2025)
- [ATLAS Navigator: Active Task-driven LAnguage-embedded Gaussian Splatting](https://mlsanity.com/paper/2502.20386/) (2025)
- [Tru-POMDP: Task Planning Under Uncertainty via Tree of Hypotheses and Open-Ended POMDPs](https://mlsanity.com/paper/2506.02860/) (2025)
- [Heterogeneous Robot Collaboration in Unstructured Environments with Grounded Generative Intelligence](https://mlsanity.com/paper/2510.26915/) (2025)
- [Vision-Language-Policy Model for Dynamic Robot Task Planning](https://mlsanity.com/paper/2512.19178/) (2025)
- [Large-Language-Model-Guided State Estimation for Partially Observable Task and Motion Planning](https://mlsanity.com/paper/2603.03704/) (2026)
- [Robot Planning and Situation Handling with Active Perception](https://mlsanity.com/paper/2604.26988/) (2026)

## Built on by (most cited first, 1 of 1)

- [In-Context Learning for Robots: Methods and Applications](https://mlsanity.com/paper/2609.36012/) (2026)

## Abstract (arXiv)

Foundation models provide robots with the ability to interpret natural language and reason about environmental context, yet most language-conditioned policies assume that goals are well-specified and that task-relevant information is provided upfront via a prior map. Operating in unfamiliar environments with underspecified tasks entails high contextual uncertainty: the robot must jointly infer what constitutes task success, what constitutes relevant information, and where (or whether) that information exists. We address these limitations via CLUE (Closed-Loop contextual Uncertainty rEsolution), a framework for actively resolving contextual uncertainty given underspecified tasks in natural language. CLUE uses an LLM-derived policy to hypothesize task-relevant concepts and potential plans. It then uses a language-embedded map, which is constructed online, to ground these hypotheses into actions. The policy sequentially evaluates hypotheses via closed-loop environment interaction and refines its plans as it gathers new information. We deploy CLUE on a Boston Dynamics Spot across three real indoor and outdoor environments spanning 15 tasks that require object disambiguation, functional inference, and occlusion reasoning. CLUE achieves a success rate within 7 percentage points of an oracle policy and outperforms an LLM-enabled planner without closed-loop feedback by a 4x margin. Supporting experiments demonstrate that simply building and then querying a language-enriched map is insufficient to resolve complex contextual planning tasks; these approaches achieve roughly one third the success rate of CLUE while requiring over 10x more VLM tokens. We provide additional information at https://zacravichandran.github.io/CLUE.