# SGDraw: Scene Graph Drawing Interface Using Object-Oriented Representation

arXiv: https://arxiv.org/abs/2211.16697 · HTML: https://mlsanity.com/paper/2211.16697/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 26 of its 36 references that are in the map.
- First posted on arXiv 30 Nov 2022.

## Builds on (most cited first, 26 of 26)

- [Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations](https://mlsanity.com/paper/1602.07332/) (2016)
- [Scene Graph Generation by Iterative Message Passing](https://mlsanity.com/paper/1701.02426/) (2017)
- [Neural Motifs: Scene Graph Parsing with Global Context](https://mlsanity.com/paper/1711.06640/) (2017)
- [Unbiased Scene Graph Generation from Biased Training](https://mlsanity.com/paper/2002.11949/) (2020)
- [Visual Relationship Detection with Language Priors](https://mlsanity.com/paper/1608.00187/) (2016)
- [Image Generation from Scene Graphs](https://mlsanity.com/paper/1804.01622/) (2018)
- [Graph R-CNN for Scene Graph Generation](https://mlsanity.com/paper/1808.00191/) (2018)
- [Scene Graph Generation from Objects, Phrases and Region Captions](https://mlsanity.com/paper/1707.09700/) (2017)
- [Energy-Based Learning for Scene Graph Generation](https://mlsanity.com/paper/2103.02221/) (2021)
- [Explainable and Explicit Visual Reasoning over Scene Graphs](https://mlsanity.com/paper/1812.01855/) (2018)
- [Graphical Contrastive Losses for Scene Graph Parsing](https://mlsanity.com/paper/1903.02728/) (2019)
- [Detecting Visual Relationships with Deep Relational Networks](https://mlsanity.com/paper/1704.03114/) (2017)
- [Say As You Wish: Fine-grained Control of Image Caption Generation with Abstract Scene Graphs](https://mlsanity.com/paper/2003.00387/) (2020)
- [Factorizable Net: An Efficient Subgraph-based Framework for Scene Graph Generation](https://mlsanity.com/paper/1806.11538/) (2018)
- [Attentive Relational Networks for Mapping Images to Scene Graphs](https://mlsanity.com/paper/1811.10696/) (2018)
- [Scene Graph Generation: A Comprehensive Survey](https://mlsanity.com/paper/2201.00443/) (2022)
- [Scene Graph Reasoning for Visual Question Answering](https://mlsanity.com/paper/2007.01072/) (2020)
- [Graph Density-Aware Losses for Novel Compositions in Scene Graph Generation](https://mlsanity.com/paper/2005.08230/) (2020)
- [Deep Variation-structured Reinforcement Learning for Visual Relationship and Attribute Detection](https://mlsanity.com/paper/1703.03054/) (2017)
- [Reasoning with Scene Graphs for Robot Planning under Partial Observability](https://mlsanity.com/paper/2202.10432/) (2022)
- [Detecting Visual Relationships Using Box Attention](https://mlsanity.com/paper/1807.02136/) (2018)
- [Generative Compositional Augmentations for Scene Graph Prediction](https://mlsanity.com/paper/2007.05756/) (2020)
- [ViP-CNN: Visual Phrase Guided Convolutional Neural Network](https://mlsanity.com/paper/1702.07191/) (2017)
- [dualFace:Two-Stage Drawing Guidance for Freehand Portrait Sketching](https://mlsanity.com/paper/2104.12297/) (2021)
- [DualSmoke: Sketch-Based Smoke Illustration Design with Two-Stage Generative Model](https://mlsanity.com/paper/2208.10906/) (2022)
- [GeneAnnotator: A Semi-automatic Annotation Tool for Visual Scene Graph](https://mlsanity.com/paper/2109.02226/) (2021)

## Abstract (arXiv)

Scene understanding is an essential and challenging task in computer vision. To provide the visually fundamental graphical structure of an image, the scene graph has received increased attention due to its powerful semantic representation. However, it is difficult to draw a proper scene graph for image retrieval, image generation, and multi-modal applications. The conventional scene graph annotation interface is not easy to use in image annotations, and the automatic scene graph generation approaches using deep neural networks are prone to generate redundant content while disregarding details. In this work, we propose SGDraw, a scene graph drawing interface using object-oriented scene graph representation to help users draw and edit scene graphs interactively. For the proposed object-oriented representation, we consider the objects, attributes, and relationships of objects as a structural unit. SGDraw provides a web-based scene graph annotation and generation tool for scene understanding applications. To verify the effectiveness of the proposed interface, we conducted a comparison study with the conventional tool and the user experience study. The results show that SGDraw can help generate scene graphs with richer details and describe the images more accurately than traditional bounding box annotations. We believe the proposed SGDraw can be useful in various vision tasks, such as image retrieval and generation.