# VHAKG: A Multi-modal Knowledge Graph Based on Synchronized Multi-view Videos of Daily Activities

arXiv: https://arxiv.org/abs/2408.14895 · HTML: https://mlsanity.com/paper/2408.14895/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 27 references that are in the map.
- First posted on arXiv 27 Aug 2024.

## Builds on (most cited first, 6 of 6)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Multi-Modal Knowledge Graph Construction and Application: A Survey](https://mlsanity.com/paper/2202.05786/) (2022)
- [Temporal Knowledge Graph Completion: A Survey](https://mlsanity.com/paper/2201.08236/) (2022)
- [What is Event Knowledge Graph: A Survey](https://mlsanity.com/paper/2112.15280/) (2021)
- [Synthesizing Event-centric Knowledge Graphs of Daily Activities Using Virtual Space](https://mlsanity.com/paper/2307.16206/) (2023)
- [VisionKG: Unleashing the Power of Visual Datasets via Knowledge Graph](https://mlsanity.com/paper/2309.13610/) (2023)

## Built on by (most cited first, 1 of 1)

- [HOME-KGQA: A Benchmark Dataset for Multimodal Knowledge Graph Question Answering on Household Daily Activities](https://mlsanity.com/paper/2605.09348/) (2026)

## Abstract (arXiv)

Multi-modal knowledge graphs (MMKGs), which ground various non-symbolic data (e.g., images and videos) into symbols, have attracted attention as resources enabling knowledge processing and machine learning across modalities. However, the construction of MMKGs for videos consisting of multiple events, such as daily activities, is still in the early stages. In this paper, we construct an MMKG based on synchronized multi-view simulated videos of daily activities. Besides representing the content of daily life videos as event-centric knowledge, our MMKG also includes frame-by-frame fine-grained changes, such as bounding boxes within video frames. In addition, we provide support tools for querying our MMKG. As an application example, we demonstrate that our MMKG facilitates benchmarking vision-language models by providing the necessary vision-language datasets for a tailored task.