# Dynamic Graph Generation Network: Generating Relational Knowledge from Diagrams

arXiv: https://arxiv.org/abs/1711.09528 · HTML: https://mlsanity.com/paper/1711.09528/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Nov 2017.

## Built on by (most cited first, 3 of 3)

- [Figuring out Figures: Using Textual References to Caption Scientific Figures](https://mlsanity.com/paper/2407.11008/) (2024)
- [DiagramRAG: A Lightweight Framework to Retrieve Scientific Diagram for Figure Generation](https://mlsanity.com/paper/2605.27931/) (2026)
- [Semantic Segmentation of Node and Edge Diagrams for Assistive Technology](https://mlsanity.com/paper/2606.11320/) (2026)

## Abstract (arXiv)

In this work, we introduce a new algorithm for analyzing a diagram, which contains visual and textual information in an abstract and integrated way. Whereas diagrams contain richer information compared with individual image-based or language-based data, proper solutions for automatically understanding them have not been proposed due to their innate characteristics of multi-modality and arbitrariness of layouts. To tackle this problem, we propose a unified diagram-parsing network for generating knowledge from diagrams based on an object detector and a recurrent neural network designed for a graphical structure. Specifically, we propose a dynamic graph-generation network that is based on dynamic memory and graph theory. We explore the dynamics of information in a diagram with activation of gates in gated recurrent unit (GRU) cells. On publicly available diagram datasets, our model demonstrates a state-of-the-art result that outperforms other baselines. Moreover, further experiments on question answering shows potentials of the proposed method for various applications.