# CityDreamer: Compositional Generative Model of Unbounded 3D Cities

arXiv: https://arxiv.org/abs/2309.00610 · HTML: https://mlsanity.com/paper/2309.00610/

## Key facts

- Cited by 76 papers in the map (data through 14 Oct 2026).
- Builds on 50 of its 65 references that are in the map.
- First posted on arXiv 1 Sep 2023.

## Builds on (most cited first, 30 of 50)

- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [Taming Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2012.09841/) (2020)
- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfer](https://mlsanity.com/paper/1907.01341/) (2019)
- [Zero-1-to-3: Zero-shot One Image to 3D Object](https://mlsanity.com/paper/2303.11328/) (2023)
- [Generating Diverse High-Fidelity Images with VQ-VAE-2](https://mlsanity.com/paper/1906.00446/) (2019)
- [Semantic Image Synthesis with Spatially-Adaptive Normalization](https://mlsanity.com/paper/1903.07291/) (2019)
- [Demystifying MMD GANs](https://mlsanity.com/paper/1801.01401/) (2018)
- [Efficient Geometry-aware 3D Generative Adversarial Networks](https://mlsanity.com/paper/2112.07945/) (2021)
- [MaskGIT: Masked Generative Image Transformer](https://mlsanity.com/paper/2202.04200/) (2022)
- [The Replica Dataset: A Digital Replica of Indoor Spaces](https://mlsanity.com/paper/1906.05797/) (2019)
- [Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling](https://mlsanity.com/paper/1610.07584/) (2016)
- [StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesis](https://mlsanity.com/paper/2110.08985/) (2021)
- [GET3D: A Generative Model of High Quality 3D Textured Shapes Learned from Images](https://mlsanity.com/paper/2209.11163/) (2022)
- [3D-FRONT: 3D Furnished Rooms with layOuts and semaNTics](https://mlsanity.com/paper/2011.09127/) (2020)
- [OmniObject3D: Large-Vocabulary 3D Object Dataset for Realistic Perception, Reconstruction and Generation](https://mlsanity.com/paper/2301.07525/) (2023)
- [RealFusion: 360deg Reconstruction of Any Object from a Single Image](https://mlsanity.com/paper/2302.10663/) (2023)
- [StyleSDF: High-Resolution 3D-Consistent Image and Geometry Generation](https://mlsanity.com/paper/2112.11427/) (2021)
- [Infinite Nature: Perpetual View Generation of Natural Scenes from a Single Image](https://mlsanity.com/paper/2012.09855/) (2020)
- [DreamBooth3D: Subject-Driven Text-to-3D Generation](https://mlsanity.com/paper/2303.13508/) (2023)
- [UnFlow: Unsupervised Learning of Optical Flow with a Bidirectional Census Loss](https://mlsanity.com/paper/1711.07837/) (2017)
- [ATISS: Autoregressive Transformers for Indoor Scene Synthesis](https://mlsanity.com/paper/2110.03675/) (2021)
- [Pix2Vox: Context-aware 3D Reconstruction from Single and Multi-view Images](https://mlsanity.com/paper/1901.11153/) (2019)
- [SceneFormer: Indoor Scene Generation with Transformers](https://mlsanity.com/paper/2012.09793/) (2020)
- [GAUDI: A Neural Architect for Immersive 3D Scene Generation](https://mlsanity.com/paper/2207.13751/) (2022)
- [Unconstrained Scene Generation with Locally Conditioned Radiance Fields](https://mlsanity.com/paper/2104.00670/) (2021)

## Built on by (most cited first, 30 of 76)

- [DriveDreamer: Towards Real-world-driven World Models for Autonomous Driving](https://mlsanity.com/paper/2309.09777/) (2023)
- [WonderWorld: Interactive 3D Scene Generation from a Single Image](https://mlsanity.com/paper/2406.09394/) (2024)
- [XCube: Large-Scale 3D Generative Modeling using Sparse Voxel Hierarchies](https://mlsanity.com/paper/2312.03806/) (2023)
- [ADriver-I: A General World Model for Autonomous Driving](https://mlsanity.com/paper/2311.13549/) (2023)
- [3DTopia: Large Text-to-3D Generation Model with Hybrid Diffusion Priors](https://mlsanity.com/paper/2403.02234/) (2024)
- [InfiniCube: Unbounded and Controllable Dynamic 3D Driving Scene Generation with World-Guided Video Models](https://mlsanity.com/paper/2412.03934/) (2024)
- [Pyramid Diffusion for Fine 3D Large Scene Generation](https://mlsanity.com/paper/2311.12085/) (2023)
- [Generative Gaussian Splatting for Unbounded 3D City Generation](https://mlsanity.com/paper/2406.06526/) (2024)
- [SynCity: Training-Free Generation of 3D Worlds](https://mlsanity.com/paper/2503.16420/) (2025)
- [A Comprehensive Survey on 3D Content Generation](https://mlsanity.com/paper/2402.01166/) (2024)
- [LetsGo: Large-Scale Garage Modeling and Rendering via LiDAR-Assisted Gaussian Primitives](https://mlsanity.com/paper/2404.09748/) (2024)
- [CityCraft: A Real Crafter for 3D City Generation](https://mlsanity.com/paper/2406.04983/) (2024)
- [StarGen: A Spatiotemporal Autoregression Framework with Video Diffusion Model for Scalable and Controllable Scene Generation](https://mlsanity.com/paper/2501.05763/) (2025)
- [FirePlace: Geometric Refinements of LLM Common Sense Reasoning for 3D Object Placement](https://mlsanity.com/paper/2503.04919/) (2025)
- [UrbanWorld: An Urban World Model for 3D City Generation](https://mlsanity.com/paper/2407.11965/) (2024)
- [3D Scene Generation: A Survey](https://mlsanity.com/paper/2505.05474/) (2025)
- [CityGen: Infinite and Controllable City Layout Generation](https://mlsanity.com/paper/2312.01508/) (2023)
- [CityX: Controllable Procedural Content Generation for Unbounded 3D Cities](https://mlsanity.com/paper/2407.17572/) (2024)
- [Lodge++: High-quality and Long Dance Generation with Vivid Choreography Patterns](https://mlsanity.com/paper/2410.20389/) (2024)
- [X-Scene: Large-Scale Driving Scene Generation with High Fidelity and Flexible Controllability](https://mlsanity.com/paper/2506.13558/) (2025)
- [NuiScene: Exploring Efficient Generation of Unbounded Outdoor Scenes](https://mlsanity.com/paper/2503.16375/) (2025)
- [Urban Architect: Steerable 3D Urban Scene Generation with Layout Prior](https://mlsanity.com/paper/2404.06780/) (2024)
- [Compositional Generative Model of Unbounded 4D Cities](https://mlsanity.com/paper/2501.08983/) (2025)
- [Skyfall-GS: Synthesizing Immersive 3D Urban Scenes from Satellite Imagery](https://mlsanity.com/paper/2510.15869/) (2025)
- [Towards Autonomous Micromobility through Scalable Urban Simulation](https://mlsanity.com/paper/2505.00690/) (2025)
- [Sat2City: 3D City Generation from A Single Satellite Image with Cascaded Latent Diffusion](https://mlsanity.com/paper/2507.04403/) (2025)
- [WorldGrow: Generating Infinite 3D World](https://mlsanity.com/paper/2510.21682/) (2025)
- [Perceptual-GS: Scene-adaptive Perceptual Densification for Gaussian Splatting](https://mlsanity.com/paper/2506.12400/) (2025)
- [COHO: Context-Sensitive City-Scale Hierarchical Urban Layout Generation](https://mlsanity.com/paper/2407.11294/) (2024)
- [Controllable 3D Outdoor Scene Generation via Scene Graphs](https://mlsanity.com/paper/2503.07152/) (2025)

## Abstract (arXiv)

3D city generation is a desirable yet challenging task, since humans are more sensitive to structural distortions in urban environments. Additionally, generating 3D cities is more complex than 3D natural scenes since buildings, as objects of the same class, exhibit a wider range of appearances compared to the relatively consistent appearance of objects like trees in natural scenes. To address these challenges, we propose CityDreamer, a compositional generative model designed specifically for unbounded 3D cities. Our key insight is that 3D city generation should be a composition of different types of neural fields: 1) various building instances, and 2) background stuff, such as roads and green lands. Specifically, we adopt the bird's eye view scene representation and employ a volumetric render for both instance-oriented and stuff-oriented neural fields. The generative hash grid and periodic positional embedding are tailored as scene parameterization to suit the distinct characteristics of building instances and background stuff. Furthermore, we contribute a suite of CityGen Datasets, including OSM and GoogleEarth, which comprises a vast amount of real-world city imagery to enhance the realism of the generated 3D cities both in their layouts and appearances. CityDreamer achieves state-of-the-art performance not only in generating realistic 3D cities but also in localized editing within the generated cities.