mlsanityData to 7 Oct 2026

Embodied Question Answering in Photorealistic Environments with Point Cloud Perception

arXiv 1904.03461 ↗First posted 6 Apr 2019cs.CV cs.AI cs.CLcited by 103

Not on the map. Why?

Abstract (arXiv)

To help bridge the gap between internet vision-style problems and the goal of vision for embodied perception we instantiate a large-scale navigation task -- Embodied Question Answering [1] in photo-realistic environments (Matterport 3D). We thoroughly study navigation policies that utilize 3D point clouds, RGB images, or their combination. Our analysis of these models reveals several key findings. We find that two seemingly naive navigation baselines, forward-only and random, are strong navigators and challenging to outperform, due to the specific choice of the evaluation setting presented by [1]. We find a novel loss-weighting scheme we call Inflection Weighting to be important when training recurrent models for navigation with behavior cloning and are able to out perform the baselines with this technique. We find that point clouds provide a richer signal than RGB images for learning obstacle avoidance, motivating the use (and continued study) of 3D deep learning models for embodied navigation.

Key facts. Cited by 103 papers in the map (data through 7 Oct 2026). First posted on arXiv 6 Apr 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2019: 1; 2020: 0; 2021: 17; 2022: 16; 2023: 20; 2024: 19; 2025: 19; 2026: 11.

103 papers.

Sort:

By area:

  1. 2021ScanQA: 3D Question Answering for Spatial Scene Understandingcited by 340
  2. 2022SQA3D: Situated Question Answering in 3D Scenescited by 303
  3. 2024A Survey on Vision-Language-Action Models for Embodied AIcited by 274
  4. 2023Point-Bind & Point-LLM: Aligning Point Cloud with Multi-modality for 3D Understanding, Generation, and Instruction Followingcited by 192
  5. 2024Aligning Cyber Space with Physical World: A Comprehensive Survey on Embodied AIcited by 175
  6. 2022PONI: Potential Functions for ObjectGoal Navigation with Interaction-free Learningcited by 149
  7. 2023Towards Learning a Generalist Model for Embodied Navigationcited by 135
  8. 2022Vision-and-Language Navigation: A Survey of Tasks, Methods, and Future Directionscited by 116
  9. 2022Habitat-Web: Learning Embodied Object-Search Strategies from Human Demonstrations at Scalecited by 116
  10. 2019Help, Anna! Visual Navigation with Natural Multimodal Assistance via Retrospective Curiosity-Encouraging Imitation Learningcited by 100
  11. 2023PIRLNav: Pretraining with Imitation and RL Finetuning for ObjectNavcited by 91
  12. 2023Talk2BEV: Language-enhanced Bird's-eye View Maps for Autonomous Drivingcited by 88
38 more
  1. 2021Pano-AVQA: Grounded Audio-Visual Question Answering on 360^circ Videoscited by 86
  2. 2023An Outlook into the Future of Egocentric Visioncited by 84
  3. 2024Explore until Confident: Efficient Exploration for Embodied Question Answeringcited by 74
  4. 2023Bird's-Eye-View Scene Graph for Vision-Language Navigationcited by 71
  5. 2023OVRL-V2: A simple state-of-art baseline for ImageNav and ObjectNavcited by 68
  6. 2024ReMEmbR: Building and Reasoning Over Long-Horizon Spatio-Temporal Memory for Robot Navigationcited by 67
  7. 2025Large VLM-based Vision-Language-Action Models for Robotic Manipulation: A Surveycited by 66
  8. 2022Housekeep: Tidying Virtual Households using Commonsense Reasoningcited by 57
  9. 20213D Question Answeringcited by 41
  10. 2021Catch Me If You Hear Me: Audio-Visual Navigation in Complex Unmapped Environments with Moving Soundscited by 40
  11. 2024CityNav: A Large-Scale Dataset for Real-World Aerial Navigationcited by 40
  12. 2024GraphEQA: Using 3D Semantic Scene Graphs for Real-time Embodied Question Answeringcited by 37
  13. 2022Iterative Vision-and-Language Navigationcited by 34
  14. 2021Refer-it-in-RGBD: A Bottom-up Approach for 3D Visual Grounding in RGBD Imagescited by 33
  15. 2021Vision-Language Navigation: A Survey and Taxonomycited by 33
  16. 2022Episodic Memory Question Answeringcited by 32
  17. 2022ScanEnts3D: Exploiting Phrase-to-3D-Object Correspondences for Improved Visio-Linguistic Models in 3D Scenescited by 32
  18. 2025CityEQA: A Hierarchical LLM Agent on Embodied Question Answering Benchmark in City Spacecited by 31
  19. 2021Core Challenges in Embodied Vision-Language Planningcited by 30
  20. 2025Beyond the Destination: A Novel Benchmark for Exploration-Aware Embodied Question Answeringcited by 29
  21. 2021The Surprising Effectiveness of Visual Odometry Techniques for Embodied PointGoal Navigationcited by 27
  22. 2022One Step at a Time: Long-Horizon Vision-and-Language Navigation with Milestonescited by 26
  23. 2022AVLEN: Audio-Visual-Language Embodied Navigation in 3D Environmentscited by 26
  24. 2024SPARTUN3D: Situated Spatial Understanding of 3D World in Large Language Modelscited by 26
  25. 2021Look Wide and Interpret Twice: Improving Performance on Interactive Instruction-following Taskscited by 25
  26. 2024ImageNet3D: Towards General-Purpose Object-Level 3D Understandingcited by 25
  27. 20253DLLM-Mem: Long-Term Spatial-Temporal Memory for Embodied 3D Large Language Modelcited by 25
  28. 2021A Survey on Human-aware Robot Navigationcited by 23
  29. 2024RoomTour3D: Geometry-Aware Video-Instruction Tuning for Embodied Navigationcited by 23
  30. 2021GridToPix: Training Embodied Agents with Minimal Supervisioncited by 20
  31. 2023360BEV: Panoramic Semantic Mapping for Indoor Bird's-Eye Viewcited by 18
  32. 2023TriVol: Point Cloud Rendering via Triple Volumescited by 16
  33. 2023Object Goal Navigation with Recursive Implicit Mapscited by 16
  34. 2024EfficientEQA: An Efficient Approach to Open-Vocabulary Embodied Question Answeringcited by 15
  35. 2021DVD: A Diagnostic Dataset for Multi-step Reasoning in Video Grounded Dialoguecited by 13
  36. 2023CoT3DRef: Chain-of-Thoughts Data-Efficient 3D Visual Groundingcited by 13
  37. 2026Explore with Long-term Memory: A Benchmark and Multimodal LLM-based Reinforcement Learning Framework for Embodied Explorationcited by 13
  38. 2025MoMa-Kitchen: A 100K+ Benchmark for Affordance-Grounded Last-Mile Navigation in Mobile Manipulationcited by 12

50 most cited of 103 All 103 (JSON)