# HoliCity: A City-Scale Data Platform for Learning Holistic 3D Structures

arXiv: https://arxiv.org/abs/2008.03286 · HTML: https://mlsanity.com/paper/2008.03286/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Aug 2020.

## Built on by (most cited first, 29 of 29)

- [Capturing, Reconstructing, and Simulating: the UrbanScene3D Dataset](https://mlsanity.com/paper/2107.04286/) (2021)
- [Editable Scene Simulation for Autonomous Driving via Collaborative LLM-Agents](https://mlsanity.com/paper/2402.05746/) (2024)
- [CityDreamer: Compositional Generative Model of Unbounded 3D Cities](https://mlsanity.com/paper/2309.00610/) (2023)
- [InfiniCity: Infinite-Scale City Synthesis](https://mlsanity.com/paper/2301.09637/) (2023)
- [Extreme Rotation Estimation using Dense Correlation Volumes](https://mlsanity.com/paper/2104.13530/) (2021)
- [Neural Light Field Estimation for Street Scenes with Differentiable Virtual Object Insertion](https://mlsanity.com/paper/2208.09480/) (2022)
- [CTRL-C: Camera calibration TRansformer with Line-Classification](https://mlsanity.com/paper/2109.02259/) (2021)
- [Sat2Scene: 3D Urban Scene Generation from Satellite Images with Diffusion](https://mlsanity.com/paper/2401.10786/) (2024)
- [Fully Convolutional Line Parsing](https://mlsanity.com/paper/2104.11207/) (2021)
- [3D Scene Generation: A Survey](https://mlsanity.com/paper/2505.05474/) (2025)
- [OmniCity: Omnipotent City Understanding with Multi-level and Multi-view Images](https://mlsanity.com/paper/2208.00928/) (2022)
- [LightSim: Neural Lighting Simulation for Urban Scenes](https://mlsanity.com/paper/2312.06654/) (2023)
- [Compositional Generative Model of Unbounded 4D Cities](https://mlsanity.com/paper/2501.08983/) (2025)
- [Enhancing Diffusion Models with 3D Perspective Geometry Constraints](https://mlsanity.com/paper/2312.00944/) (2023)
- [SceneSplat++: A Large Dataset and Comprehensive Benchmark for Language Gaussian Splatting](https://mlsanity.com/paper/2506.08710/) (2025)
- [Augmenting Depth Estimation with Geospatial Context](https://mlsanity.com/paper/2109.09879/) (2021)
- [Sat3DGen: Comprehensive Street-Level 3D Scene Generation from Single Satellite Image](https://mlsanity.com/paper/2605.14984/) (2026)
- [Satellite to GroundScape -- Large-scale Consistent Ground View Generation from Satellite Views](https://mlsanity.com/paper/2504.15786/) (2025)
- [Deep Single Image Camera Calibration by Heatmap Regression to Recover Fisheye Images Under Manhattan World Assumption](https://mlsanity.com/paper/2303.17166/) (2023)
- [AutoEncoding Tree for City Generation and Applications](https://mlsanity.com/paper/2309.15941/) (2023)
- [SOFI: Multi-Scale Deformable Transformer for Camera Calibration with Enhanced Line Queries](https://mlsanity.com/paper/2409.15553/) (2024)
- [Disrupting Hierarchical Reasoning: Adversarial Protection for Geographic Privacy in Multimodal Reasoning Models](https://mlsanity.com/paper/2512.08503/) (2025)
- [Sat2City v2: Native 3D City Asset Generation from a Single Satellite Image](https://mlsanity.com/paper/2606.24138/) (2026)
- [ControlVP: Interactive Geometric Refinement of AI-Generated Images with Consistent Vanishing Points](https://mlsanity.com/paper/2512.07504/) (2025)
- [Spheriverse: 3D Scene Understanding from Spherical Observations in the Wild](https://mlsanity.com/paper/2609.09012/) (2026)
- [Estimating Extreme 3D Image Rotation with Transformer Cross-Attention](https://mlsanity.com/paper/2303.02615/) (2023)
- [Bird Eye-View to Street-View: A Survey](https://mlsanity.com/paper/2405.08961/) (2024)
- [Fine-Grained Building Function Recognition from Street-View Images via Geometry-Aware Semi-Supervised Learning](https://mlsanity.com/paper/2408.09460/) (2024)
- [GridFlow: Structured Latent Flow for Seamless City-Scale 3D Point Cloud Generation](https://mlsanity.com/paper/2608.29793/) (2026)

## Abstract (arXiv)

We present HoliCity, a city-scale 3D dataset with rich structural information. Currently, this dataset has 6,300 real-world panoramas of resolution 13312 × 6656 that are accurately aligned with the CAD model of downtown London with an area of more than 20 km2, in which the median reprojection error of the alignment of an average image is less than half a degree. This dataset aims to be an all-in-one data platform for research of learning abstracted high-level holistic 3D structures that can be derived from city CAD models, e.g., corners, lines, wireframes, planes, and cuboids, with the ultimate goal of supporting real-world applications including city-scale reconstruction, localization, mapping, and augmented reality. The accurate alignment of the 3D CAD models and panoramas also benefits low-level 3D vision tasks such as surface normal estimation, as the surface normal extracted from previous LiDAR-based datasets is often noisy. We conduct experiments to demonstrate the applications of HoliCity, such as predicting surface segmentation, normal maps, depth maps, and vanishing points, as well as test the generalizability of methods trained on HoliCity and other related datasets. HoliCity is available at https://holicity.io.