mlsanityData to 13 Oct 2026

Embodied Question Answering

arXiv 1711.11543 ↗First posted 30 Nov 2017cs.CV cs.AI cs.CLcited by 335

Not on the map. Why?

Cited by 38 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We present a new AI task -- Embodied Question Answering (EmbodiedQA) -- where an agent is spawned at a random location in a 3D environment and asked a question ("What color is the car?"). In order to answer, the agent must first intelligently navigate to explore the environment, gather information through first-person (egocentric) vision, and then answer the question ("orange"). This challenging task requires a range of AI skills -- active perception, language understanding, goal-driven navigation, commonsense reasoning, and grounding of language into actions. In this work, we develop the environments, end-to-end-trained reinforcement learning agents, and evaluation protocols for EmbodiedQA.

Key facts. Cited by 335 papers in the map (data through 13 Oct 2026). First posted on arXiv 30 Nov 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2018: 2; 2019: 2; 2020: 0; 2021: 59; 2022: 61; 2023: 51; 2024: 51; 2025: 55; 2026: 54.

335 papers.

Sort:

By area:

  1. 2021ScanQA: 3D Question Answering for Spatial Scene Understandingcited by 341
  2. 2022SQA3D: Situated Question Answering in 3D Scenescited by 306
  3. 2018Reinforced Cross-Modal Matching and Self-Supervised Imitation Learning for Vision-Language Navigationcited by 304
  4. 2024A Survey on Vision-Language-Action Models for Embodied AIcited by 277
  5. 2023NuScenes-QA: A Multi-modal Visual Question Answering Benchmark for Autonomous Driving Scenariocited by 262
  6. 2021A Survey of Embodied AI: From Simulators to Research Taskscited by 258
  7. 2023Habitat 3.0: A Co-Habitat for Humans, Avatars and Robotscited by 221
  8. 2022Think Global, Act Local: Dual-scale Graph Transformer for Vision-and-Language Navigationcited by 188
  9. 2024Aligning Cyber Space with Physical World: A Comprehensive Survey on Embodied AIcited by 177
  10. 2021BEHAVIOR: Benchmark for Everyday Household Activities in Virtual, Interactive, and Ecological Environmentscited by 163
  11. 2022HUMANISE: Language-conditioned Human Motion Generation in 3D Scenescited by 157
  12. 2022PONI: Potential Functions for ObjectGoal Navigation with Interaction-free Learningcited by 149
38 more
  1. 2024CycleResearcher: Improving Automated Research via Automated Reviewcited by 145
  2. 2021MultiBench: Multiscale Benchmarks for Multimodal Representation Learningcited by 144
  3. 2023Towards Learning a Generalist Model for Embodied Navigationcited by 136
  4. 2022Bridging the Gap Between Learning in Discrete and Continuous Environments for Vision-and-Language Navigationcited by 129
  5. 2023MobileVLM : A Fast, Strong and Open Vision Language Assistant for Mobile Devicescited by 120
  6. 2021FILM: Following Instructions in Language with Modular Methodscited by 119
  7. 2021SOON: Scenario Oriented Object Navigation with Graph-based Explorationcited by 118
  8. 2022Vision-and-Language Navigation: A Survey of Tasks, Methods, and Future Directionscited by 116
  9. 2022Habitat-Web: Learning Embodied Object-Search Strategies from Human Demonstrations at Scalecited by 116
  10. 2021Visual Room Rearrangementcited by 101
  11. 2019Help, Anna! Visual Navigation with Natural Multimodal Assistance via Retrospective Curiosity-Encouraging Imitation Learningcited by 100
  12. 2021Skill Induction and Planning with Latent Languagecited by 99
  13. 2022Exploiting Unlabeled Data with Vision and Language Models for Object Detectioncited by 97
  14. 2024GOAT-Bench: A Benchmark for Multi-Modal Lifelong Navigationcited by 93
  15. 2021LanguageRefer: Spatial-Language Model for 3D Visual Groundingcited by 88
  16. 2023Do Embodied Agents Dream of Pixelated Sheep: Embodied Decision Making using Language Guided World Modellingcited by 88
  17. 2021Pano-AVQA: Grounded Audio-Visual Question Answering on 360^circ Videoscited by 86
  18. 2023An Outlook into the Future of Egocentric Visioncited by 84
  19. 2024Move as You Say, Interact as You Can: Language-guided Human Motion Generation with Scene Affordancecited by 80
  20. 2018Vision-based Navigation with Language-based Assistance via Imitation Learning with Indirect Interventioncited by 77
  21. 2024Explore until Confident: Efficient Exploration for Embodied Question Answeringcited by 74
  22. 2023Octopus: Embodied Vision-Language Programmer from Environmental Feedbackcited by 73
  23. 2022VLMbench: A Compositional Benchmark for Vision-and-Language Manipulationcited by 68
  24. 2023ARNOLD: A Benchmark for Language-Grounded Task Learning With Continuous States in Realistic 3D Scenescited by 68
  25. 2023Learning to Model the World with Languagecited by 68
  26. 2025Large VLM-based Vision-Language-Action Models for Robotic Manipulation: A Surveycited by 68
  27. 2019From Language to Goals: Inverse Reinforcement Learning for Vision-Based Instruction Followingcited by 67
  28. 2024ReMEmbR: Building and Reasoning Over Long-Horizon Spatio-Temporal Memory for Robot Navigationcited by 67
  29. 2021The ThreeDWorld Transport Challenge: A Visually Guided Task-and-Motion Planning Benchmark for Physically Realistic Embodied AIcited by 65
  30. 2021Memory-Augmented Reinforcement Learning for Image-Goal Navigationcited by 64
  31. 2022DialFRED: Dialogue-Enabled Agents for Embodied Instruction Followingcited by 63
  32. 2022A Dataset for Interactive Vision-Language Navigation with Unknown Command Feasibilitycited by 61
  33. 20233D Concept Learning and Reasoning from Multi-View Imagescited by 60
  34. 2021Embodied BERT: A Transformer Model for Embodied, Language-guided Visual Task Completioncited by 58
  35. 2022Housekeep: Tidying Virtual Households using Commonsense Reasoningcited by 57
  36. 2025EmbodiedEval: Evaluate Multimodal LLMs as Embodied Agentscited by 56
  37. 2023CLIP-Guided Vision-Language Pre-training for Question Answering in 3D Scenescited by 55
  38. 2023See and Think: Embodied Agent in Virtual Environmentcited by 54

50 most cited of 335 All 335 (JSON)