ViewMind3D: Modular View-Aware Inference for Training-Free 3D-QA
On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models
Abstract (arXiv)
Recent advances in large language models (LLMs) and vision-language models (VLMs) have enabled new possibilities for 3D question answering (3D-QA), a key capability for embodied AI and robotic perception. However, most existing methods rely on 3D-specific training or fine-tuning with costly annotations, limiting their scalability and real-world applicability. We present ViewMind3D, a fully training-free and modular framework for 3D spatial reasoning over multi-view observations of a scene without requiring complete 3D reconstruction. The framework decomposes the 3D-QA task into four interpretable components: (1) question-driven multi-view selection, (2) guided visual grounding with language-conditioned object cues, (3) spatial context encoding via a bird's-eye-view (BEV) viewpoint indicator, and (4) structured answer generation through role-based reasoning. This design enables structured, robust, and interpretable reasoning without requiring model tuning. Experimental results on ScanQA and SQA3D show that ViewMind3D achieves competitive performance compared to prior training-free and fine-tuned 3D-LLMs. In particular, our method improves performance on spatially grounded question types, such as ``What'' questions in SQA3D, while maintaining strong overall accuracy (50.8%) and achieving 73.4 CIDEr on ScanQA. These results demonstrate that effective 3D reasoning can be achieved through modular orchestration of general-purpose LLMs and VLMs for robotic perception in real-world environments.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 22 of its 28 references that are in the map. First posted on arXiv 30 Jul 2026.
Builds on
22 of 28 references.
Closest ancestors · 4
Same direction, newest first
- 2026Chat-Scene++: Exploiting Context-Rich Object Identification for 3D LLM29 Mar
- 2025GPT4Scene: Understand 3D Scenes from Videos with Vision-Language Models2 Jan
- 2024ShapeLLM: Universal 3D Object Understanding for Embodied Interaction27 Feb
- 2023LL3DA: Visual Interactive Instruction Tuning for Omni-3D Understanding, Reasoning, and Planning30 Nov
Other references · 18
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2023Visual Instruction Tuningcited by 7,792
- 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
- 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
- 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
- 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
- 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,088
- 2014CIDEr: Consensus-based Image Description Evaluationcited by 2,027
- 2019ScanRefer: 3D Object Localization in RGB-D Scans using Natural Languagecited by 446
- 2023Florence-2: Advancing a Unified Representation for a Variety of Vision Taskscited by 403
- 2021ScanQA: 3D Question Answering for Spatial Scene Understandingcited by 340
- 2023PointLLM: Empowering Large Language Models to Understand Point Cloudscited by 326
6 more
- 2022SQA3D: Situated Question Answering in 3D Scenescited by 303
- 2023Chat-Scene: Bridging 3D Scene and Large Language Models with Object Identifierscited by 170
- 2023LLM-Grounder: Open-Vocabulary 3D Visual Grounding with Large Language Model as an Agentcited by 134
- 2024Grounded 3D-LLM with Referent Tokenscited by 111
- 2025DSPNet: Dual-vision Scene Perception for Robust 3D Question Answeringcited by 9
- 2025Advancing 3D Scene Understanding with MV-ScanQA Multi-View Reasoning Evaluation and TripAlign Pre-training Datasetcited by 8
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026CoV: Chain-of-View Prompting for Spatial Reasoning
- 20233D Concept Learning and Reasoning from Multi-View Images
- 2025Embodied Intelligence for 3D Understanding: A Survey on 3D Scene Question Answering
- 2025Spatial Understanding from Videos: Structured Prompts Meet Simulation Data
- 2025N3D-VLM: Native 3D Grounding Enables Accurate Spatial Reasoning in Vision-Language Models
- 20253D Question Answering via only 2D Vision-Language Models
- 2026Do 3D Large Language Models Really Understand 3D Spatial Relationships?
- 2026Dense Reward for Multi-View 3D Reasoning with Global Maps and Local Views