# HOME-KGQA: A Benchmark Dataset for Multimodal Knowledge Graph Question Answering on Household Daily Activities

arXiv: https://arxiv.org/abs/2605.09348 · HTML: https://mlsanity.com/paper/2605.09348/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 18 of its 45 references that are in the map.
- First posted on arXiv 10 May 2026.

## Builds on (most cited first, 18 of 18)

- [VirtualHome: Simulating Household Activities via Programs](https://mlsanity.com/paper/1806.07011/) (2018)
- [The Web as a Knowledge-base for Answering Complex Questions](https://mlsanity.com/paper/1803.06643/) (2018)
- [Think-on-Graph: Deep and Responsible Reasoning of Large Language Model on Knowledge Graph](https://mlsanity.com/paper/2307.07697/) (2023)
- [Key-Value Memory Networks for Directly Reading Documents](https://mlsanity.com/paper/1606.03126/) (2016)
- [Beyond I.I.D.: Three Levels of Generalization for Question Answering on Knowledge Bases](https://mlsanity.com/paper/2011.07743/) (2020)
- [Variational Reasoning for Question Answering with Knowledge Graph](https://mlsanity.com/paper/1709.04071/) (2017)
- [Knowledge-Augmented Language Model Prompting for Zero-Shot Knowledge Graph Question Answering](https://mlsanity.com/paper/2306.04136/) (2023)
- [KQA Pro: A Dataset with Explicit Compositional Programs for Complex Question Answering over Knowledge Base](https://mlsanity.com/paper/2007.03875/) (2020)
- [Multi-Modal Knowledge Graph Construction and Application: A Survey](https://mlsanity.com/paper/2202.05786/) (2022)
- [Can ChatGPT Replace Traditional KBQA Models? An In-depth Analysis of the Question Answering Performance of the GPT LLM Family](https://mlsanity.com/paper/2303.07992/) (2023)
- [Modern Baselines for SPARQL Semantic Parsing](https://mlsanity.com/paper/2204.12793/) (2022)
- [SOSA: A Lightweight Ontology for Sensors, Observations, Samples, and Actuators](https://mlsanity.com/paper/1805.09979/) (2018)
- [Event-QA: A Dataset for Event-Centric Question Answering over Knowledge Graphs](https://mlsanity.com/paper/2004.11861/) (2020)
- [EventKG - the Hub of Event Knowledge on the Web - and Biographical Timeline Generation](https://mlsanity.com/paper/1905.08794/) (2019)
- [Knowledge Graph Question Answering Datasets and Their Generalizability: Are They Enough for Future Research?](https://mlsanity.com/paper/2205.06573/) (2022)
- [SPARQL Query Generation with LLMs: Measuring the Impact of Training Data Memorization and Knowledge Injection](https://mlsanity.com/paper/2507.13859/) (2025)
- [Synthesizing Event-centric Knowledge Graphs of Daily Activities Using Virtual Space](https://mlsanity.com/paper/2307.16206/) (2023)
- [VHAKG: A Multi-modal Knowledge Graph Based on Synchronized Multi-view Videos of Daily Activities](https://mlsanity.com/paper/2408.14895/) (2024)

## Abstract (arXiv)

Large Language Models (LLMs) provide flexible natural language processing capabilities, while knowledge graphs (KGs) offer explicit and structured knowledge. Integrating these two in a complementary manner enables the development of reliable and verifiable AI systems. In particular, knowledge graph question answering (KGQA) has attracted attention as a means to reduce LLM hallucinations and to leverage knowledge beyond the training data. However, existing KGQA benchmark datasets are biased toward encyclopedic knowledge, limited to a single modality, and lack fine-grained spatiotemporal data, which limits their applicability to real-world scenarios targeted by Embodied AI. We introduce HOME-KGQA, a novel KGQA benchmark dataset built on a multimodal KG of daily household activities. HOME-KGQA consists of complex, multi-hop natural language questions paired with graph database query languages. Compared to existing benchmarks, it includes more challenging questions that involve multi-level spatiotemporal reasoning, multimodal grounding, and aggregate functions. Experimental results show that the LLM-based KGQA methods fail to achieve performance comparable to that on existing datasets when evaluated on HOME-KGQA. This highlights significant challenges that should be addressed for the real-world deployment of KGQA systems. Our dataset is available at https://github.com/aistairc/home-kgqa