# Unconditional Scene Graph Generation

arXiv: https://arxiv.org/abs/2108.05884 · HTML: https://mlsanity.com/paper/2108.05884/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 45 references that are in the map.
- First posted on arXiv 12 Aug 2021.

## Builds on (most cited first, 30 of 32)

- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [Improved Techniques for Training GANs](https://mlsanity.com/paper/1606.03498/) (2016)
- [Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations](https://mlsanity.com/paper/1602.07332/) (2016)
- [On the Properties of Neural Machine Translation: Encoder-Decoder Approaches](https://mlsanity.com/paper/1409.1259/) (2014)
- [Deep Anomaly Detection with Outlier Exposure](https://mlsanity.com/paper/1812.04606/) (2018)
- [Junction Tree Variational Autoencoder for Molecular Graph Generation](https://mlsanity.com/paper/1802.04364/) (2018)
- [Scene Graph Generation by Iterative Message Passing](https://mlsanity.com/paper/1701.02426/) (2017)
- [Neural Motifs: Scene Graph Parsing with Global Context](https://mlsanity.com/paper/1711.06640/) (2017)
- [GraphRNN: Generating Realistic Graphs with Deep Auto-regressive Models](https://mlsanity.com/paper/1802.08773/) (2018)
- [Assessing Generative Models via Precision and Recall](https://mlsanity.com/paper/1806.00035/) (2018)
- [MolGAN: An implicit generative model for small molecular graphs](https://mlsanity.com/paper/1805.11973/) (2018)
- [GraphVAE: Towards Generation of Small Graphs Using Variational Autoencoders](https://mlsanity.com/paper/1802.03480/) (2018)
- [Unbiased Scene Graph Generation from Biased Training](https://mlsanity.com/paper/2002.11949/) (2020)
- [Image Generation from Scene Graphs](https://mlsanity.com/paper/1804.01622/) (2018)
- [Community Detection and Stochastic Block Models](https://mlsanity.com/paper/1703.10146/) (2017)
- [Graph R-CNN for Scene Graph Generation](https://mlsanity.com/paper/1808.00191/) (2018)
- [A note on the evaluation of generative models](https://mlsanity.com/paper/1511.01844/) (2015)
- [Efficient Graph Generation with Graph Recurrent Attention Networks](https://mlsanity.com/paper/1910.00760/) (2019)
- [Learning Deep Generative Models of Graphs](https://mlsanity.com/paper/1803.03324/) (2018)
- [Learning 3D Semantic Scene Graphs from 3D Indoor Reconstructions](https://mlsanity.com/paper/2004.03967/) (2020)
- [Constrained Graph Variational Autoencoders for Molecule Design](https://mlsanity.com/paper/1805.09076/) (2018)
- [SceneGraphFusion: Incremental 3D Scene Graph Prediction from RGB-D Sequences](https://mlsanity.com/paper/2103.14898/) (2021)
- [NetGAN: Generating Graphs via Random Walks](https://mlsanity.com/paper/1803.00816/) (2018)
- [Meta-Sim: Learning to Generate Synthetic Datasets](https://mlsanity.com/paper/1904.11621/) (2019)
- [Graphite: Iterative Generative Modeling of Graphs](https://mlsanity.com/paper/1803.10459/) (2018)
- [Semantic Image Manipulation Using Scene Graphs](https://mlsanity.com/paper/2004.03677/) (2020)
- [Factorizable Net: An Efficient Subgraph-based Framework for Scene Graph Generation](https://mlsanity.com/paper/1806.11538/) (2018)
- [Specifying Object Attributes and Relations in Interactive Scene Generation](https://mlsanity.com/paper/1909.05379/) (2019)
- [Pixels to Graphs by Associative Embedding](https://mlsanity.com/paper/1706.07365/) (2017)
- [SceneGraphNet: Neural Message Passing for 3D Indoor Scene Augmentation](https://mlsanity.com/paper/1907.11308/) (2019)

## Built on by (most cited first, 22 of 22)

- [InstructScene: Instruction-Driven 3D Indoor Scene Synthesis with Semantic Graph Prior](https://mlsanity.com/paper/2402.04717/) (2024)
- [SceneGenie: Scene Graph Guided Diffusion Models for Image Synthesis](https://mlsanity.com/paper/2304.14573/) (2023)
- [Incremental 3D Semantic Scene Graph Prediction from RGB Sequences](https://mlsanity.com/paper/2305.02743/) (2023)
- [Segment Anything Model for Road Network Graph Extraction](https://mlsanity.com/paper/2403.16051/) (2024)
- [MIGS: Meta Image Generation from Scene Graphs](https://mlsanity.com/paper/2110.11918/) (2021)
- [InstructLayout: Instruction-Driven 2D and 3D Layout Synthesis with Semantic Graph Prior](https://mlsanity.com/paper/2407.07580/) (2024)
- [DisPositioNet: Disentangled Pose and Identity in Semantic Image Manipulation](https://mlsanity.com/paper/2211.05499/) (2022)
- [Joint Generative Modeling of Grounded Scene Graphs and Images via Diffusion Models](https://mlsanity.com/paper/2401.01130/) (2024)
- [GEMS: Scene Expansion using Generative Models of Graphs](https://mlsanity.com/paper/2207.03729/) (2022)
- [Generalized Unbiased Scene Graph Generation](https://mlsanity.com/paper/2308.04802/) (2023)
- [PRISM: Progressive Restoration for Scene Graph-based Image Manipulation](https://mlsanity.com/paper/2311.02247/) (2023)
- [Instance-incremental Scene Graph Generation from Real-world Point Clouds via Normalizing Flows](https://mlsanity.com/paper/2302.10425/) (2023)
- [LIC-GAN: Language Information Conditioned Graph Generative GAN Model](https://mlsanity.com/paper/2306.01937/) (2023)
- [Biasing Like Human: A Cognitive Bias Framework for Scene Graph Generation](https://mlsanity.com/paper/2203.09160/) (2022)
- [Integrating Symbolic Reasoning into Neural Generative Models for Design Generation](https://mlsanity.com/paper/2310.09383/) (2023)
- [Maps from Motion (MfM): Generating 2D Semantic Maps from Sparse Multi-view Images](https://mlsanity.com/paper/2411.12620/) (2024)
- [FORESCENE: FOREcasting human activity via latent SCENE graphs diffusion](https://mlsanity.com/paper/2503.06182/) (2025)
- [Generated Contents Enrichment](https://mlsanity.com/paper/2405.03650/) (2024)
- [GraphSCENE: On-Demand Critical Scenario Generation for Autonomous Vehicles in Simulation](https://mlsanity.com/paper/2410.13514/) (2024)
- [BUSSARD: Normalizing Flows for Bijective Universal Scene-Specific Anomalous Relationship Detection](https://mlsanity.com/paper/2603.16645/) (2026)
- [Can We Build Scene Graphs, Not Classify Them? FlowSG: Progressive Image-Conditioned Scene Graph Generation with Flow Matching](https://mlsanity.com/paper/2604.18623/) (2026)
- [Dependency-Aware Discrete Diffusion for Scene Graph Generation](https://mlsanity.com/paper/2605.09065/) (2026)

## Abstract (arXiv)

Despite recent advancements in single-domain or single-object image generation, it is still challenging to generate complex scenes containing diverse, multiple objects and their interactions. Scene graphs, composed of nodes as objects and directed-edges as relationships among objects, offer an alternative representation of a scene that is more semantically grounded than images. We hypothesize that a generative model for scene graphs might be able to learn the underlying semantic structure of real-world scenes more effectively than images, and hence, generate realistic novel scenes in the form of scene graphs. In this work, we explore a new task for the unconditional generation of semantic scene graphs. We develop a deep auto-regressive model called SceneGraphGen which can directly learn the probability distribution over labelled and directed graphs using a hierarchical recurrent architecture. The model takes a seed object as input and generates a scene graph in a sequence of steps, each step generating an object node, followed by a sequence of relationship edges connecting to the previous nodes. We show that the scene graphs generated by SceneGraphGen are diverse and follow the semantic patterns of real-world scenes. Additionally, we demonstrate the application of the generated graphs in image synthesis, anomaly detection and scene graph completion.