mlsanityData to 7 Oct 2026

Embodied Intelligence for 3D Understanding: A Survey on 3D Scene Question Answering

arXiv 2502.00342 ↗First posted 1 Feb 2025cs.CVcited by 11

Not on the map. Why?

Abstract (arXiv)

3D Scene Question Answering (3D SQA) represents an interdisciplinary task that integrates 3D visual perception and natural language processing, empowering intelligent agents to comprehend and interact with complex 3D environments. Recent advances in large multimodal modelling have driven the creation of diverse datasets and spurred the development of instruction-tuning and zero-shot methods for 3D SQA. However, this rapid progress introduces challenges, particularly in achieving unified analysis and comparison across datasets and baselines. In this survey, we provide the first comprehensive and systematic review of 3D SQA. We organize existing work from three perspectives: datasets, methodologies, and evaluation metrics. Beyond basic categorization, we identify shared architectural patterns across methods. Our survey further synthesizes core limitations and discusses how current trends, such as instruction tuning, multimodal alignment, and zero-shot, can shape future developments. Finally, we propose a range of promising research directions covering dataset construction, task generalization, interaction modeling, and unified evaluation protocols. This work aims to serve as a foundation for future research and foster progress toward more generalizable and intelligent 3D SQA systems.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 75 of its 112 references that are in the map. First posted on arXiv 1 Feb 2025.

Builds on

75 of 112 references.

References · 75

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  4. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  5. 2020Language Models are Few-Shot Learnerscited by 32,199
  6. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  7. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  8. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  9. 2017Mask R-CNNcited by 7,259
  10. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  11. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
  12. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,384
38 more
  1. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  2. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  3. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  4. 2015VQA: Visual Question Answeringcited by 2,489
  5. 2014CIDEr: Consensus-based Image Description Evaluationcited by 2,027
  6. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,973
  7. 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,522
  8. 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoningcited by 1,449
  9. 2016CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoningcited by 1,445
  10. 2019Habitat: A Platform for Embodied AI Researchcited by 1,163
  11. 2017AI2-THOR: An Interactive 3D Environment for Visual AIcited by 850
  12. 2016SPICE: Semantic Propositional Image Caption Evaluationcited by 820
  13. 2022Language-driven Semantic Segmentationcited by 711
  14. 2023The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision)cited by 677
  15. 2021Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AIcited by 579
  16. 2019Deep Learning for 3D Point Clouds: A Surveycited by 557
  17. 2022OpenScene: 3D Scene Understanding with Open Vocabulariescited by 556
  18. 2021Habitat 2.0: Training Home Assistants to Rearrange their Habitatcited by 501
  19. 2018Gibson Env: Real-World Perception for Embodied Agentscited by 477
  20. 2016Semantic Scene Completion from a Single Depth Imagecited by 468
  21. 2019ScanRefer: 3D Object Localization in RGB-D Scans using Natural Languagecited by 446
  22. 2021ARKitScenes: A Diverse Real-World Dataset For 3D Indoor Scene Understanding Using Mobile RGB-D Datacited by 442
  23. 2023An Embodied Generalist Agent in 3D Worldcited by 355
  24. 2019Point-Voxel CNN for Efficient 3D Deep Learningcited by 343
  25. 2021ScanQA: 3D Question Answering for Spatial Scene Understandingcited by 340
  26. 2017Embodied Question Answeringcited by 335
  27. 2022SQA3D: Situated Question Answering in 3D Scenescited by 303
  28. 2022Mask3D: Mask Transformer for 3D Semantic Instance Segmentationcited by 299
  29. 2019Resolving 3D Human Pose Ambiguities with 3D Scene Constraintscited by 294
  30. 2020Scan2Cap: Context-aware Dense Captioning in RGB-D Scanscited by 215
  31. 2021Group-Free 3D Object Detection via Transformerscited by 215
  32. 2021Stochastic Scene-Aware Motion Predictioncited by 203
  33. 2023Scalable 3D Captioning with Pretrained Modelscited by 201
  34. 20233D-VisTA: Pre-trained Transformer for 3D Vision and Text Alignmentcited by 201
  35. 2019RIO: 3D Object Instance Re-Localization in Changing Indoor Environmentscited by 187
  36. 2023Chat-Scene: Bridging 3D Scene and Large Language Models with Object Identifierscited by 170
  37. 2023LAMM: Language-Assisted Multi-Modal Instruction-Tuning Dataset, Framework, and Benchmarkcited by 169
  38. 2017IQA: Visual Question Answering in Interactive Environmentscited by 154

50 most cited of 75 All 75 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 6; 2026: 5.

11 papers.

Similar, not cited

Close in wording, no citation either way

  1. 20253D-MoRe: Unified Modal-Contextual Reasoning for Embodied Question Answering
  2. 2025HIS-GPT: Towards 3D Human-In-Scene Multimodal Understanding
  3. 2026ViewMind3D: Modular View-Aware Inference for Training-Free 3D-QA
  4. 2024GraphEQA: Using 3D Semantic Scene Graphs for Real-time Embodied Question Answering
  5. 2025GraphPad: Inference-Time 3D Scene Graph Updates for Embodied Question Answering
  6. 2026CoV: Chain-of-View Prompting for Spatial Reasoning
  7. 2025SpatialReasoner: Active Perception for Large-Scale 3D Scene Understanding
  8. 2024Agent3D-Zero: An Agent for Zero-shot 3D Understanding