3D Instruction Ambiguity Detection
Not on the map. Why?
Abstract (arXiv)
In safety-critical domains, linguistic ambiguity can have severe consequences; a vague command like "Pass me the vial" in a surgical setting could lead to catastrophic errors. Yet, most embodied AI research overlooks this, assuming instructions are clear and focusing on execution rather than confirmation. To address this critical safety gap, we are the first to define 3D Instruction Ambiguity Detection, a fundamental new task where a model must determine if a command has a single, unambiguous meaning within a given 3D scene. To support this research, we build Ambi3D, the large-scale benchmark for this task, featuring over 700 diverse 3D scenes and around 22k instructions. Our analysis reveals a surprising limitation: state-of-the-art 3D Large Language Models (LLMs) struggle to reliably determine if an instruction is ambiguous. To address this challenge, we propose AmbiVer, a two-stage framework that collects explicit visual evidence from multiple views and uses it to guide an vision-language model (VLM) in judging instruction ambiguity. Extensive experiments demonstrate the challenge of our task and the effectiveness of AmbiVer, paving the way for safer and more trustworthy embodied AI. Code and dataset available at https://jiayuding031020.github.io/ambi3d/.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 14 of its 33 references that are in the map. First posted on arXiv 9 Jan 2026.
Builds on
14 of 33 references.
References · 14
- 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
- 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,510
- 2021MDETR -- Modulated Detection for End-to-End Multi-Modal Understandingcited by 819
- 2021ScanQA: 3D Question Answering for Spatial Scene Understandingcited by 346
- 20233D-VisTA: Pre-trained Transformer for 3D Vision and Text Alignmentcited by 204
- 2024LLaVA-3D: A Simple yet Effective Pathway to Empowering LMMs with 3D-awarenesscited by 183
- 2023Chat-Scene: Bridging 3D Scene and Large Language Models with Object Identifierscited by 175
- 2020Referring Expression Comprehension: A Survey of Methods and Datasetscited by 79
- 2024LSceneLLM: Enhancing Large 3D Scene Understanding Using Adaptive Visual Preferencescited by 38
- 2023Think, Act, and Ask: Open-World Interactive Personalized Robot Navigationcited by 31
- 2024Robin3D: Improving 3D Large Language Model via Robust Instruction Tuningcited by 26
- 2019Enabling Robots to Understand Incomplete Natural Language Instructions Using Commonsense Reasoningcited by 13
Built on by
Citing papers per year: 2026: 3.
3 papers.
Similar, not cited
Close in wording, no citation either way
- 2025LLM-based ambiguity detection in natural language instructions for collaborative surgical robots
- 2023M3DBench: Let's Instruct Large Models with Multi-modal 3D Prompts
- 2024Visual-O1: Understanding Ambiguous Instructions via Multi-modal Multi-turn Chain-of-thoughts Reasoning
- 20243DBench: A Scalable 3D Benchmark and Instruction-Tuning Dataset
- 2026The Yes-Man Syndrome: Benchmarking Abstention in Embodied Robotic Agents
- 2026GuardianBench: A Same-Scene Instruction-Contrastive Benchmark for Latent Contextual Risk in Embodied AI
- 2025Masking Matters: Unlocking the Spatial Reasoning Capabilities of LLMs for 3D Scene-Language Understanding
- 2023M3IT: A Large-Scale Dataset towards Multi-Modal Multilingual Instruction Tuning