mlsanityData to 7 Oct 2026

Multi-Target Embodied Question Answering

arXiv 1904.04686 ↗First posted 9 Apr 2019cs.CVcited by 74

Not on the map. Why?

Abstract (arXiv)

Embodied Question Answering (EQA) is a relatively new task where an agent is asked to answer questions about its environment from egocentric perception. EQA makes the fundamental assumption that every question, e.g., "what color is the car?", has exactly one target ("car") being inquired about. This assumption puts a direct limitation on the abilities of the agent. We present a generalization of EQA - Multi-Target EQA (MT-EQA). Specifically, we study questions that have multiple targets in them, such as "Is the dresser in the bedroom bigger than the oven in the kitchen?", where the agent has to navigate to multiple locations ("dresser in bedroom", "oven in kitchen") and perform comparative reasoning ("dresser" bigger than "oven") before it can answer a question. Such questions require the development of entirely new modules or components in the agent. To address this, we propose a modular architecture composed of a program generator, a controller, a navigator, and a VQA module. The program generator converts the given question into sequential executable sub-programs; the navigator guides the agent to multiple locations pertinent to the navigation-related sub-programs; and the controller learns to select relevant observations along its path. These observations are then fed to the VQA module to predict the answer. We perform detailed analysis for each of the model components and show that our joint model can outperform previous methods and strong baselines by a significant margin.

Key facts. Cited by 74 papers in the map (data through 7 Oct 2026). First posted on arXiv 9 Apr 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 10; 2022: 10; 2023: 11; 2024: 14; 2025: 22; 2026: 7.

74 papers.

Sort:

By area:

  1. 2021ScanQA: 3D Question Answering for Spatial Scene Understandingcited by 340
  2. 2023Tool Learning with Foundation Modelscited by 302
  3. 2024A Survey on Vision-Language-Action Models for Embodied AIcited by 274
  4. 2021A Survey of Embodied AI: From Simulators to Research Taskscited by 256
  5. 2022Think Global, Act Local: Dual-scale Graph Transformer for Vision-and-Language Navigationcited by 187
  6. 2024Aligning Cyber Space with Physical World: A Comprehensive Survey on Embodied AIcited by 175
  7. 2021BEHAVIOR: Benchmark for Everyday Household Activities in Virtual, Interactive, and Ecological Environmentscited by 161
  8. 2022Habitat-Web: Learning Embodied Object-Search Strategies from Human Demonstrations at Scalecited by 116
  9. 2023An Outlook into the Future of Egocentric Visioncited by 84
  10. 2024Explore until Confident: Efficient Exploration for Embodied Question Answeringcited by 74
  11. 2025Large VLM-based Vision-Language-Action Models for Robotic Manipulation: A Surveycited by 66
  12. 20233D Concept Learning and Reasoning from Multi-View Imagescited by 60
38 more
  1. 2025EmbodiedEval: Evaluate Multimodal LLMs as Embodied Agentscited by 55
  2. 2023See and Think: Embodied Agent in Virtual Environmentcited by 54
  3. 2022Learning from Unlabeled 3D Environments for Vision-and-Language Navigationcited by 49
  4. 2024Embodied-RAG: General Non-parametric Embodied Memory for Retrieval and Generationcited by 44
  5. 2021Catch Me If You Hear Me: Audio-Visual Navigation in Complex Unmapped Environments with Moving Soundscited by 40
  6. 2021Vision-Language Navigation: A Survey and Taxonomycited by 33
  7. 2022AssistQ: Affordance-centric Question-driven Task Completion for Egocentric Assistantcited by 32
  8. 2022Episodic Memory Question Answeringcited by 32
  9. 2022ScanEnts3D: Exploiting Phrase-to-3D-Object Correspondences for Improved Visio-Linguistic Models in 3D Scenescited by 32
  10. 2025CityEQA: A Hierarchical LLM Agent on Embodied Question Answering Benchmark in City Spacecited by 31
  11. 2021Core Challenges in Embodied Vision-Language Planningcited by 30
  12. 2025Beyond the Destination: A Novel Benchmark for Exploration-Aware Embodied Question Answeringcited by 29
  13. 2023Multimodality Representation Learning: A Survey on Evolution, Pretraining and Its Applicationscited by 27
  14. 2022AVLEN: Audio-Visual-Language Embodied Navigation in 3D Environmentscited by 26
  15. 20253DLLM-Mem: Long-Term Spatial-Temporal Memory for Embodied 3D Large Language Modelcited by 25
  16. 2024Hierarchical Auto-Organizing System for Open-Ended Multi-Agent Navigationcited by 24
  17. 2025Challenges and Trends in Egocentric Vision: A Surveycited by 20
  18. 2021Interpretation of Emergent Communication in Heterogeneous Collaborative Embodied Agentscited by 19
  19. 2025IR3D-Bench: Evaluating Vision-Language Model Scene Understanding as Agentic Inverse Renderingcited by 17
  20. 2025Seeing from Another Perspective: Evaluating Multi-View Understanding in MLLMscited by 16
  21. 2024EfficientEQA: An Efficient Approach to Open-Vocabulary Embodied Question Answeringcited by 15
  22. 2026Explore with Long-term Memory: A Benchmark and Multimodal LLM-based Reinforcement Learning Framework for Embodied Explorationcited by 13
  23. 2025MoMa-Kitchen: A 100K+ Benchmark for Affordance-Grounded Last-Mile Navigation in Mobile Manipulationcited by 12
  24. 2025Memory-Centric Embodied Question Answeringcited by 12
  25. 2025IndustryEQA: Pushing the Frontiers of Embodied Question Answering in Industrial Scenarioscited by 12
  26. 2025Enter the Mind Palace: Reasoning and Planning for Long-term Active Embodied Question Answeringcited by 12
  27. 2024Evaluating Zero-Shot GPT-4V Performance on 3D Visual Question Answering Benchmarkscited by 10
  28. 20243D Question Answering for City Scene Understandingcited by 10
  29. 2024NoisyEQA: Benchmarking Embodied Question Answering Against Noisy Queriescited by 10
  30. 2024Map-based Modular Approach for Zero-shot Embodied Question Answeringcited by 9
  31. 2025When Robots Should Say "I Don't Know": Benchmarking Abstention in Embodied Question Answeringcited by 9
  32. 2023Active Reasoning in an Open-World Environmentcited by 8
  33. 2022Embodied Referring Expression for Manipulation Question Answering in Interactive Environmentcited by 7
  34. 2023Multi-Level Compositional Reasoning for Interactive Instruction Followingcited by 7
  35. 2023Embodied Agents for Efficient Exploration and Smart Scene Descriptioncited by 6
  36. 2025Understanding Dynamic Scenes in Ego Centric 4D Point Cloudscited by 6
  37. 2026SpaMEM: Benchmarking Dynamic Spatial Reasoning via Perception-Memory Integration in Embodied Environmentscited by 6
  38. 2023What You Say Is What You Show: Visual Narration Detection in Instructional Videoscited by 5

50 most cited of 74 All 74 (JSON)