# ZRG: A Dataset for Multimodal 3D Residential Rooftop Understanding

arXiv: https://arxiv.org/abs/2304.13219 · HTML: https://mlsanity.com/paper/2304.13219/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 25 of its 58 references that are in the map.
- First posted on arXiv 26 Apr 2023.

## Builds on (most cited first, 25 of 25)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1802.02611/) (2018)
- [Pyramid Scene Parsing Network](https://mlsanity.com/paper/1612.01105/) (2016)
- [Vision Transformers for Dense Prediction](https://mlsanity.com/paper/2103.13413/) (2021)
- [pixelNeRF: Neural Radiance Fields from One or Few Images](https://mlsanity.com/paper/2012.02190/) (2020)
- [UNet++: A Nested U-Net Architecture for Medical Image Segmentation](https://mlsanity.com/paper/1807.10165/) (2018)
- [WILDS: A Benchmark of in-the-Wild Distribution Shifts](https://mlsanity.com/paper/2012.07421/) (2020)
- [Geometric deep learning: going beyond Euclidean data](https://mlsanity.com/paper/1611.08097/) (2016)
- [NeRF++: Analyzing and Improving Neural Radiance Fields](https://mlsanity.com/paper/2010.07492/) (2020)
- [Simple Copy-Paste is a Strong Data Augmentation Method for Instance Segmentation](https://mlsanity.com/paper/2012.07177/) (2020)
- [Remote Sensing Image Change Detection with Transformers](https://mlsanity.com/paper/2103.00208/) (2021)
- [SpaceNet: A Remote Sensing Dataset and Challenge Series](https://mlsanity.com/paper/1807.01232/) (2018)
- [xBD: A Dataset for Assessing Building Damage from Satellite Imagery](https://mlsanity.com/paper/1911.09296/) (2019)
- [Extending the WILDS Benchmark for Unsupervised Adaptation](https://mlsanity.com/paper/2112.05090/) (2021)
- [S2Looking: A Satellite Side-Looking Dataset for Building Change Detection](https://mlsanity.com/paper/2107.09244/) (2021)
- [Continental-Scale Building Detection from High Resolution Satellite Imagery](https://mlsanity.com/paper/2107.12283/) (2021)
- [IM2HEIGHT: Height Estimation from Single Monocular Imagery via Fully Residual Convolutional-Deconvolutional Network](https://mlsanity.com/paper/1802.10249/) (2018)
- [Height estimation from single aerial images using a deep ordinal regression network](https://mlsanity.com/paper/2006.02801/) (2020)
- [Intuitive and Efficient Roof Modeling for Reconstruction and Synthesis](https://mlsanity.com/paper/2109.07683/) (2021)
- [Vectorizing World Buildings: Planar Graph Reconstruction by Primitive Detection and Relationship Inference](https://mlsanity.com/paper/1912.05135/) (2019)
- [Learning to Construct 3D Building Wireframes from 3D Line Clouds](https://mlsanity.com/paper/2208.11948/) (2022)
- [Supervising Remote Sensing Change Detection Models with 3D Surface Semantics](https://mlsanity.com/paper/2202.13251/) (2022)

## Abstract (arXiv)

A crucial part of any home is the roof over our heads to protect us from the elements. In this paper we present the Zeitview Rooftop Geometry (ZRG) dataset for residential rooftop understanding. ZRG is a large-scale residential rooftop dataset of over 20k properties collected through roof inspections from across the U.S. and contains multiple modalities including high resolution aerial orthomosaics, digital surface models (DSM), colored point clouds, and 3D roof wireframe annotations. We provide an in-depth analysis and perform several experimental baselines including roof outline extraction, monocular height estimation, and planar roof structure extraction, to illustrate a few of the numerous potential applications unlocked by this dataset.