# Synthetic 3D Data Generation Pipeline for Geometric Deep Learning in Architecture

arXiv: https://arxiv.org/abs/2104.12564 · HTML: https://mlsanity.com/paper/2104.12564/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 25 references that are in the map.
- First posted on arXiv 26 Apr 2021.

## Builds on (most cited first, 14 of 14)

- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [Matterport3D: Learning from RGB-D Data in Indoor Environments](https://mlsanity.com/paper/1709.06158/) (2017)
- [The Replica Dataset: A Digital Replica of Indoor Spaces](https://mlsanity.com/paper/1906.05797/) (2019)
- [Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understanding](https://mlsanity.com/paper/2011.02523/) (2020)
- [PartNet: A Large-scale Benchmark for Fine-grained and Hierarchical Part-level 3D Object Understanding](https://mlsanity.com/paper/1812.02713/) (2018)
- [Gibson Env: Real-World Perception for Embodied Agents](https://mlsanity.com/paper/1808.10654/) (2018)
- [ABC: A Big CAD Model Dataset For Geometric Deep Learning](https://mlsanity.com/paper/1812.06216/) (2018)
- [Training Deep Networks with Synthetic Data: Bridging the Reality Gap by Domain Randomization](https://mlsanity.com/paper/1804.06516/) (2018)
- [Structured3D: A Large Photo-realistic Dataset for Structured 3D Modeling](https://mlsanity.com/paper/1908.00222/) (2019)
- [Pix3D: Dataset and Methods for Single-Image 3D Shape Modeling](https://mlsanity.com/paper/1804.04610/) (2018)
- [InteriorNet: Mega-scale Multi-sensor Photo-realistic Indoor Scenes Dataset](https://mlsanity.com/paper/1809.00716/) (2018)
- [Learning 3D Part Assembly from a Single Image](https://mlsanity.com/paper/2003.09754/) (2020)
- [SceneNet: Understanding Real World Indoor Scenes With Synthetic Data](https://mlsanity.com/paper/1511.07041/) (2015)
- [SUM: A Benchmark Dataset of Semantic Urban Meshes](https://mlsanity.com/paper/2103.00355/) (2021)

## Built on by (most cited first, 7 of 7)

- [Vitruvio: 3D Building Meshes via Single Perspective Sketches](https://mlsanity.com/paper/2210.13634/) (2022)
- [Exploiting Semantic Scene Reconstruction for Estimating Building Envelope Characteristics](https://mlsanity.com/paper/2410.22383/) (2024)
- [Deep Sketch-Based 3D Modeling: A Survey](https://mlsanity.com/paper/2603.03287/) (2026)
- [Deep Aramaic: Towards a Synthetic Data Paradigm Enabling Machine Learning in Epigraphy](https://mlsanity.com/paper/2310.07310/) (2023)
- [Multi-task 3D building understanding with multi-modal pretraining](https://mlsanity.com/paper/2306.10146/) (2023)
- [Novel Synthetic Data Tool for Data-Driven Cardboard Box Localization](https://mlsanity.com/paper/2305.05215/) (2023)
- [DC3DO: Diffusion Classifier for 3D Objects](https://mlsanity.com/paper/2408.06693/) (2024)

## Abstract (arXiv)

With the growing interest in deep learning algorithms and computational design in the architectural field, the need for large, accessible and diverse architectural datasets increases. We decided to tackle this problem by constructing a field-specific synthetic data generation pipeline that generates an arbitrary amount of 3D data along with the associated 2D and 3D annotations. The variety of annotations, the flexibility to customize the generated building and dataset parameters make this framework suitable for multiple deep learning tasks, including geometric deep learning that requires direct 3D supervision. Creating our building data generation pipeline we leveraged architectural knowledge from experts in order to construct a framework that would be modular, extendable and would provide a sufficient amount of class-balanced data samples. Moreover, we purposefully involve the researcher in the dataset customization allowing the introduction of additional building components, material textures, building classes, number and type of annotations as well as the number of views per 3D model sample. In this way, the framework would satisfy different research requirements and would be adaptable to a large variety of tasks. All code and data are made publicly available.