# Knowledge-based Embodied Question Answering

arXiv: https://arxiv.org/abs/2109.07872 · HTML: https://mlsanity.com/paper/2109.07872/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Sep 2021.

## Built on by (most cited first, 29 of 29)

- [Aligning Cyber Space with Physical World: A Comprehensive Survey on Embodied AI](https://mlsanity.com/paper/2407.06886/) (2024)
- [EDA: Explicit Text-Decoupling and Dense Alignment for 3D Visual Grounding](https://mlsanity.com/paper/2209.14941/) (2022)
- [Embodied-RAG: General Non-parametric Embodied Memory for Retrieval and Generation](https://mlsanity.com/paper/2409.18313/) (2024)
- [CityNav: A Large-Scale Dataset for Real-World Aerial Navigation](https://mlsanity.com/paper/2406.14240/) (2024)
- [Beyond the Destination: A Novel Benchmark for Exploration-Aware Embodied Question Answering](https://mlsanity.com/paper/2503.11117/) (2025)
- [Generative Multi-Agent Collaboration in Embodied AI: A Systematic Review](https://mlsanity.com/paper/2502.11518/) (2025)
- [IndustryEQA: Pushing the Frontiers of Embodied Question Answering in Industrial Scenarios](https://mlsanity.com/paper/2505.20640/) (2025)
- [FROSS: Faster-than-Real-Time Online 3D Semantic Scene Graph Generation from RGB-D Images](https://mlsanity.com/paper/2507.19993/) (2025)
- [When Robots Should Say "I Don't Know": Benchmarking Abstention in Embodied Question Answering](https://mlsanity.com/paper/2512.04597/) (2025)
- [GeReA: Question-Aware Prompt Captions for Knowledge-based Visual Question Answering](https://mlsanity.com/paper/2402.02503/) (2024)
- [Map-based Modular Approach for Zero-shot Embodied Question Answering](https://mlsanity.com/paper/2405.16559/) (2024)
- [Heterogeneous Embodied Multi-Agent Collaboration](https://mlsanity.com/paper/2307.13957/) (2023)
- [Synthesizing Event-centric Knowledge Graphs of Daily Activities Using Virtual Space](https://mlsanity.com/paper/2307.16206/) (2023)
- [MEIA: Multimodal Embodied Perception and Interaction in Unknown Environments](https://mlsanity.com/paper/2402.00290/) (2024)
- [A VLM-based Method for Visual Anomaly Detection in Robotic Scientific Laboratories](https://mlsanity.com/paper/2506.05405/) (2025)
- [ReEXplore: Improving MLLMs for Embodied Exploration with Contextualized Retrospective Experience Replay](https://mlsanity.com/paper/2511.19033/) (2025)
- [Is the House Ready For Sleeptime? Generating and Evaluating Situational Queries for Embodied Question Answering](https://mlsanity.com/paper/2405.04732/) (2024)
- [AssistantX: An LLM-Powered Proactive Assistant in Collaborative Human-Populated Environment](https://mlsanity.com/paper/2409.17655/) (2024)
- [Structured Observation Language for Efficient and Generalizable Vision-Language Navigation](https://mlsanity.com/paper/2603.27577/) (2026)
- [R4: Retrieval-Augmented Reasoning for Vision-Language Models in 4D Spatio-Temporal Space](https://mlsanity.com/paper/2512.15940/) (2025)
- [ActiveFly-Bench: Aligning Embodied Question Answering with Vision-Language-Action for Aerial Embodied Perception](https://mlsanity.com/paper/2607.10180/) (2026)
- [Augmented Vision-Language Models: A Systematic Review](https://mlsanity.com/paper/2507.22933/) (2025)
- [Memory-Guided View Refinement for Dynamic Human-in-the-loop EQA](https://mlsanity.com/paper/2603.09541/) (2026)
- [Modernising Reinforcement Learning-Based Navigation for Embodied Semantic Scene Graph Generation](https://mlsanity.com/paper/2603.25415/) (2026)
- [Model Adaptation for Time Constrained Embodied Control](https://mlsanity.com/paper/2406.11128/) (2024)
- [ConEQsA: Concurrent and Asynchronous Embodied Questions Scheduling and Answering](https://mlsanity.com/paper/2509.11663/) (2025)
- [IndustryAssetEQA: A Neurosymbolic Operational Intelligence System for Embodied Question Answering in Industrial Asset Maintenance](https://mlsanity.com/paper/2604.23446/) (2026)
- [AeroVerse-SatAgent: UAV-Satellite Collaborative Spatial Reasoning Inspired by the Dual Visual Pathway Theory of Cognitive Neuroscience](https://mlsanity.com/paper/2606.31467/) (2026)
- [Advantage-Guided Gate: Reshaping Open-Ended Reasoning for Vision-Based Spatial Intelligence](https://mlsanity.com/paper/2608.07987/) (2026)

## Abstract (arXiv)

In this paper, we propose a novel Knowledge-based Embodied Question Answering (K-EQA) task, in which the agent intelligently explores the environment to answer various questions with the knowledge. Different from explicitly specifying the target object in the question as existing EQA work, the agent can resort to external knowledge to understand more complicated question such as "Please tell me what are objects used to cut food in the room?", in which the agent must know the knowledge such as "knife is used for cutting food". To address this K-EQA problem, a novel framework based on neural program synthesis reasoning is proposed, where the joint reasoning of the external knowledge and 3D scene graph is performed to realize navigation and question answering. Especially, the 3D scene graph can provide the memory to store the visual information of visited scenes, which significantly improves the efficiency for the multi-turn question answering. Experimental results have demonstrated that the proposed framework is capable of answering more complicated and realistic questions in the embodied environment. The proposed method is also applicable to multi-agent scenarios.