# Generating Triples with Adversarial Networks for Scene Graph Construction

arXiv: https://arxiv.org/abs/1802.02598 · HTML: https://mlsanity.com/paper/1802.02598/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Feb 2018.

## Built on by (most cited first, 5 of 5)

- [Scene Graph Generation: A Comprehensive Survey](https://mlsanity.com/paper/2201.00443/) (2022)
- [Discrete Graph Auto-Encoder](https://mlsanity.com/paper/2306.07735/) (2023)
- [Towards Book Cover Design via Layout Graphs](https://mlsanity.com/paper/2105.11088/) (2021)
- [GEMS: Scene Expansion using Generative Models of Graphs](https://mlsanity.com/paper/2207.03729/) (2022)
- [Hierarchical Discrete Flow Matching for Graph Generation](https://mlsanity.com/paper/2604.00236/) (2026)

## Abstract (arXiv)

Driven by successes in deep learning, computer vision research has begun to move beyond object detection and image classification to more sophisticated tasks like image captioning or visual question answering. Motivating such endeavors is the desire for models to capture not only objects present in an image, but more fine-grained aspects of a scene such as relationships between objects and their attributes. Scene graphs provide a formal construct for capturing these aspects of an image. Despite this, there have been only a few recent efforts to generate scene graphs from imagery. Previous works limit themselves to settings where bounding box information is available at train time and do not attempt to generate scene graphs with attributes. In this paper we propose a method, based on recent advancements in Generative Adversarial Networks, to overcome these deficiencies. We take the approach of first generating small subgraphs, each describing a single statement about a scene from a specific region of the input image chosen using an attention mechanism. By doing so, our method is able to produce portions of the scene graphs with attribute information without the need for bounding box labels. Then, the complete scene graph is constructed from these subgraphs. We show that our model improves upon prior work in scene graph generation on state-of-the-art data sets and accepted metrics. Further, we demonstrate that our model is capable of handling a larger vocabulary size than prior work has attempted.