# Hierarchical Knowledge Graphs for Story Understanding in Visual Narratives

arXiv: https://arxiv.org/abs/2506.10008 · HTML: https://mlsanity.com/paper/2506.10008/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Apr 2025.

## Built on by (most cited first, 5 of 5)

- [GameTileNet: A Semantic Dataset for Low-Resolution Game Art in Procedural Content Generation](https://mlsanity.com/paper/2507.02941/) (2025)
- [Narrative-to-Scene Generation: An LLM-Driven Pipeline for 2D Game Environments](https://mlsanity.com/paper/2509.04481/) (2025)
- [Robust Symbolic Reasoning for Visual Narratives via Hierarchical and Semantically Normalized Knowledge Graphs](https://mlsanity.com/paper/2508.14941/) (2025)
- [Reconstructing Persistent Worlds from Narratives for Narrative-Grounded Interactive Experiences](https://mlsanity.com/paper/2608.04037/) (2026)
- [Distinguishing Revision and Delayed Elaboration in Incremental Narrative Interpretation](https://mlsanity.com/paper/2608.21364/) (2026)

## Abstract (arXiv)

We present a hierarchical knowledge graph framework for the structured semantic understanding of visual narratives, using comics as a representative domain for multimodal storytelling. The framework organizes narrative content across three levels-panel, event, and macro-event, by integrating symbolic graphs that encode semantic, spatial, and temporal relationships. At the panel level, it models visual elements such as characters, objects, and actions alongside textual components including dialogue and narration. These are systematically connected to higher-level graphs that capture narrative sequences and abstract story structures. Applied to a manually annotated subset of the Manga109 dataset, the framework supports interpretable symbolic reasoning across four representative tasks: action retrieval, dialogue tracing, character appearance mapping, and timeline reconstruction. Rather than prioritizing predictive performance, the system emphasizes transparency in narrative modeling and enables structured inference aligned with cognitive theories of event segmentation and visual storytelling. This work contributes to explainable narrative analysis and offers a foundation for authoring tools, narrative comprehension systems, and interactive media applications.