# 3D Instruction Ambiguity Detection

arXiv: https://arxiv.org/abs/2601.05991 · HTML: https://mlsanity.com/paper/2601.05991/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 33 references that are in the map.
- First posted on arXiv 9 Jan 2026.

## Builds on (most cited first, 14 of 14)

- [Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection](https://mlsanity.com/paper/2303.05499/) (2023)
- [InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiency](https://mlsanity.com/paper/2508.18265/) (2025)
- [MDETR -- Modulated Detection for End-to-End Multi-Modal Understanding](https://mlsanity.com/paper/2104.12763/) (2021)
- [ScanQA: 3D Question Answering for Spatial Scene Understanding](https://mlsanity.com/paper/2112.10482/) (2021)
- [3D-VisTA: Pre-trained Transformer for 3D Vision and Text Alignment](https://mlsanity.com/paper/2308.04352/) (2023)
- [LLaVA-3D: A Simple yet Effective Pathway to Empowering LMMs with 3D-awareness](https://mlsanity.com/paper/2409.18125/) (2024)
- [Chat-Scene: Bridging 3D Scene and Large Language Models with Object Identifiers](https://mlsanity.com/paper/2312.08168/) (2023)
- [Referring Expression Comprehension: A Survey of Methods and Datasets](https://mlsanity.com/paper/2007.09554/) (2020)
- [LSceneLLM: Enhancing Large 3D Scene Understanding Using Adaptive Visual Preferences](https://mlsanity.com/paper/2412.01292/) (2024)
- [Think, Act, and Ask: Open-World Interactive Personalized Robot Navigation](https://mlsanity.com/paper/2310.07968/) (2023)
- [Robin3D: Improving 3D Large Language Model via Robust Instruction Tuning](https://mlsanity.com/paper/2410.00255/) (2024)
- [Enabling Robots to Understand Incomplete Natural Language Instructions Using Commonsense Reasoning](https://mlsanity.com/paper/1904.12907/) (2019)
- [Collaborative Instance Object Navigation: Leveraging Uncertainty-Awareness to Minimize Human-Agent Dialogues](https://mlsanity.com/paper/2412.01250/) (2024)
- [A Multimodal Classifier Generative Adversarial Network for Carry and Place Tasks from Ambiguous Language Instructions](https://mlsanity.com/paper/1806.03847/) (2018)

## Built on by (most cited first, 3 of 3)

- [ZeroSplat: Generalized Referring Segmentation in 3D Gaussian Splatting](https://mlsanity.com/paper/2607.18801/) (2026)
- [LMM-Track4D: Eliciting 4D Dynamic Reasoning in LMMs via Trajectory-Grounded Dialogue](https://mlsanity.com/paper/2605.19390/) (2026)
- [CausalSplat: Towards Comprehensive Hierarchical Reasoning in 3D Gaussian Splatting](https://mlsanity.com/paper/2608.11150/) (2026)

## Abstract (arXiv)

In safety-critical domains, linguistic ambiguity can have severe consequences; a vague command like "Pass me the vial" in a surgical setting could lead to catastrophic errors. Yet, most embodied AI research overlooks this, assuming instructions are clear and focusing on execution rather than confirmation. To address this critical safety gap, we are the first to define 3D Instruction Ambiguity Detection, a fundamental new task where a model must determine if a command has a single, unambiguous meaning within a given 3D scene. To support this research, we build Ambi3D, the large-scale benchmark for this task, featuring over 700 diverse 3D scenes and around 22k instructions. Our analysis reveals a surprising limitation: state-of-the-art 3D Large Language Models (LLMs) struggle to reliably determine if an instruction is ambiguous. To address this challenge, we propose AmbiVer, a two-stage framework that collects explicit visual evidence from multiple views and uses it to guide an vision-language model (VLM) in judging instruction ambiguity. Extensive experiments demonstrate the challenge of our task and the effectiveness of AmbiVer, paving the way for safer and more trustworthy embodied AI. Code and dataset available at https://jiayuding031020.github.io/ambi3d/.