# BuilDiff: 3D Building Shape Generation using Single-Image Conditional Point Cloud Diffusion Models

arXiv: https://arxiv.org/abs/2309.00158 · HTML: https://mlsanity.com/paper/2309.00158/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models.
- Builds on 31 of its 50 references that are in the map.
- First posted on arXiv 31 Aug 2023.

## Placed in

- Diffusion models / [3D, point cloud and scene generation](https://mlsanity.com/diffusion/3d-point-cloud-and-scene-generation/)

## Builds on (most cited first, 30 of 31)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [Variational Inference with Normalizing Flows](https://mlsanity.com/paper/1505.05770/) (2015)
- [Multi-Scale Context Aggregation by Dilated Convolutions](https://mlsanity.com/paper/1511.07122/) (2015)
- [A Point Set Generation Network for 3D Object Reconstruction from a Single Image](https://mlsanity.com/paper/1612.00603/) (2016)
- [Diffusion Probabilistic Models for 3D Point Cloud Generation](https://mlsanity.com/paper/2103.01458/) (2021)
- [Point-E: A System for Generating 3D Point Clouds from Complex Prompts](https://mlsanity.com/paper/2212.08751/) (2022)
- [3D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstruction](https://mlsanity.com/paper/1604.00449/) (2016)
- [LION: Latent Point Diffusion Models for 3D Shape Generation](https://mlsanity.com/paper/2210.06978/) (2022)
- [3D Shape Generation and Completion through Point-Voxel Diffusion](https://mlsanity.com/paper/2104.03670/) (2021)
- [PointFlow: 3D Point Cloud Generation with Continuous Normalizing Flows](https://mlsanity.com/paper/1906.12320/) (2019)
- [Point-Voxel CNN for Efficient 3D Deep Learning](https://mlsanity.com/paper/1907.03739/) (2019)
- [DISN: Deep Implicit Surface Network for High-quality Single-view 3D Reconstruction](https://mlsanity.com/paper/1905.10711/) (2019)
- [Density-aware Chamfer Distance as a Comprehensive Metric for Point Cloud Completion](https://mlsanity.com/paper/2111.12702/) (2021)
- [Multiresolution Tree Networks for 3D Point Cloud Processing](https://mlsanity.com/paper/1807.03520/) (2018)
- [PC2: Projection-Conditioned Point Cloud Diffusion for Single-Image 3D Reconstruction](https://mlsanity.com/paper/2302.10668/) (2023)
- [Towards Semantic Segmentation of Urban-Scale 3D Point Clouds: A Dataset, Benchmarks and Challenges](https://mlsanity.com/paper/2009.03137/) (2020)
- [3D Shape Reconstruction from 2D Images with Disentangled Attribute Flow](https://mlsanity.com/paper/2203.15190/) (2022)
- [BuildingNet: Learning to Label 3D Buildings](https://mlsanity.com/paper/2110.04955/) (2021)
- [Topologically-Aware Deformation Fields for Single-View 3D Reconstruction](https://mlsanity.com/paper/2205.06267/) (2022)
- [Automated 3D reconstruction of LoD2 and LoD1 models for all 10 million buildings of the Netherlands](https://mlsanity.com/paper/2201.01191/) (2021)
- [Flow-based GAN for 3D Point Cloud Generation from a Single Image](https://mlsanity.com/paper/2210.04072/) (2022)

## Built on by (most cited first, 6 of 6)

- [A Comprehensive Survey on Diffusion Models and Their Applications](https://mlsanity.com/paper/2408.10207/) (2024)
- [SAT-SKYLINES: 3D Building Generation from Satellite Imagery and Coarse Geometric Priors](https://mlsanity.com/paper/2508.18531/) (2025)
- [AIM2PC: Aerial Image to 3D Building Point Cloud Reconstruction](https://mlsanity.com/paper/2503.18527/) (2025)
- [GeoTexBuild: 3D Building Model Generation from Map Footprints](https://mlsanity.com/paper/2504.08419/) (2025)
- [Sketch2MinSurf: Vision-Language Guided Generation of Editable Minimal Surfaces from Hand-Drawn Sketches](https://mlsanity.com/paper/2605.20733/) (2026)
- [Point Diffusion Mamba: Unified Diffusion-State-Space Modeling for Single-View 3D Reconstruction under Data Scarcity](https://mlsanity.com/paper/2609.25538/) (2026)

## Abstract (arXiv)

3D building generation with low data acquisition costs, such as single image-to-3D, becomes increasingly important. However, most of the existing single image-to-3D building creation works are restricted to those images with specific viewing angles, hence they are difficult to scale to general-view images that commonly appear in practical cases. To fill this gap, we propose a novel 3D building shape generation method exploiting point cloud diffusion models with image conditioning schemes, which demonstrates flexibility to the input images. By cooperating two conditional diffusion models and introducing a regularization strategy during denoising process, our method is able to synthesize building roofs while maintaining the overall structures. We validate our framework on two newly built datasets and extensive experiments show that our method outperforms previous works in terms of building generation quality.