# Contextual Scene Augmentation and Synthesis via GSACNet

arXiv: https://arxiv.org/abs/2103.15369 · HTML: https://mlsanity.com/paper/2103.15369/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 55 references that are in the map.
- First posted on arXiv 29 Mar 2021.

## Builds on (most cited first, 26 of 26)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [Fast Graph Representation Learning with PyTorch Geometric](https://mlsanity.com/paper/1903.02428/) (2019)
- [Matterport3D: Learning from RGB-D Data in Indoor Environments](https://mlsanity.com/paper/1709.06158/) (2017)
- [Deep Graph Infomax](https://mlsanity.com/paper/1809.10341/) (2018)
- [AI2-THOR: An Interactive 3D Environment for Visual AI](https://mlsanity.com/paper/1712.05474/) (2017)
- [Link Prediction Based on Graph Neural Networks](https://mlsanity.com/paper/1802.09691/) (2018)
- [Gated Graph Sequence Neural Networks](https://mlsanity.com/paper/1511.05493/) (2015)
- [Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understanding](https://mlsanity.com/paper/2011.02523/) (2020)
- [Semantic Scene Completion from a Single Depth Image](https://mlsanity.com/paper/1611.08974/) (2016)
- [GraphRNN: Generating Realistic Graphs with Deep Auto-regressive Models](https://mlsanity.com/paper/1802.08773/) (2018)
- [Fast and Flexible Indoor Scene Synthesis via Deep Convolutional Generative Models](https://mlsanity.com/paper/1811.12463/) (2018)
- [Interpolated Convolutional Networks for 3D Point Cloud Understanding](https://mlsanity.com/paper/1908.04512/) (2019)
- [OpenRooms: An End-to-End Open Framework for Photorealistic Indoor Scene Datasets](https://mlsanity.com/paper/2007.12868/) (2020)
- [SceneGraphNet: Neural Message Passing for 3D Indoor Scene Augmentation](https://mlsanity.com/paper/1907.11308/) (2019)
- [GRAINS: Generative Recursive Autoencoders for INdoor Scenes](https://mlsanity.com/paper/1807.09193/) (2018)
- [Learning to Measure Change: Fully Convolutional Siamese Metric Networks for Scene Change Detection](https://mlsanity.com/paper/1810.09111/) (2018)
- [SceneGen: Generative Contextual Scene Augmentation using Scene Graph Priors](https://mlsanity.com/paper/2009.12395/) (2020)
- [SceneSuggest: Context-driven 3D Scene Design](https://mlsanity.com/paper/1703.00061/) (2017)
- [SketchOpt: Sketch-based Parametric Model Retrieval for Generative Design](https://mlsanity.com/paper/2009.00261/) (2020)
- [Optimization and Manipulation of Contextual Mutual Spaces for Multi-User Virtual and Augmented Reality Interaction](https://mlsanity.com/paper/1910.05998/) (2019)
- [GenScan: A Generative Method for Populating Parametric 3D Scan Datasets](https://mlsanity.com/paper/2012.03998/) (2020)
- [GenFloor: Interactive Generative Space Layout System via Encoded Tree Graphs](https://mlsanity.com/paper/2102.10320/) (2021)

## Built on by (most cited first, 2 of 2)

- [CLIP-Layout: Style-Consistent Indoor Scene Synthesis with Semantic Furniture Embedding](https://mlsanity.com/paper/2303.03565/) (2023)
- [Mutual Scene Synthesis for Mixed Reality Telepresence](https://mlsanity.com/paper/2204.00161/) (2022)

## Abstract (arXiv)

Indoor scene augmentation has become an emerging topic in the field of computer vision and graphics with applications in augmented and virtual reality. However, current state-of-the-art systems using deep neural networks require large datasets for training. In this paper we introduce GSACNet, a contextual scene augmentation system that can be trained with limited scene priors. GSACNet utilizes a novel parametric data augmentation method combined with a Graph Attention and Siamese network architecture followed by an Autoencoder network to facilitate training with small datasets. We show the effectiveness of our proposed system by conducting ablation and comparative studies with alternative systems on the Matterport3D dataset. Our results indicate that our scene augmentation outperforms prior art in scene synthesis with limited scene priors available.