# Assessing out-of-domain generalization for robust building damage detection

arXiv: https://arxiv.org/abs/2011.10328 · HTML: https://mlsanity.com/paper/2011.10328/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Nov 2020.

## Built on by (most cited first, 13 of 13)

- [Mission Critical -- Satellite Data is a Distinct Modality in Machine Learning](https://mlsanity.com/paper/2402.01444/) (2024)
- [EarthNet2021: A large-scale dataset and challenge for Earth surface forecasting as a guided video prediction task](https://mlsanity.com/paper/2104.10066/) (2021)
- [EarthShift: a benchmark for measuring robustness to real-world distribution shifts in Earth observation](https://mlsanity.com/paper/2605.29330/) (2026)
- [Exploiting Position Bias for Robust Aspect Sentiment Classification](https://mlsanity.com/paper/2105.14210/) (2021)
- [Fast building segmentation from satellite imagery and few local labels](https://mlsanity.com/paper/2206.05377/) (2022)
- [Rapid building damage assessment workflow: An implementation for the 2023 Rolling Fork, Mississippi tornado event](https://mlsanity.com/paper/2306.12589/) (2023)
- [BLDNet: A Semi-supervised Change Detection Building Damage Framework using Graph Convolutional Networks and Urban Domain Knowledge](https://mlsanity.com/paper/2201.10389/) (2022)
- [Robust Disaster Assessment from Aerial Imagery Using Text-to-Image Synthetic Data](https://mlsanity.com/paper/2405.13779/) (2024)
- [ConeQuest: A Benchmark for Cone Segmentation on Mars](https://mlsanity.com/paper/2311.08657/) (2023)
- [Building Damage Mapping with Self-PositiveUnlabeled Learning](https://mlsanity.com/paper/2111.02586/) (2021)
- [Disaster mapping from satellites: damage detection with crowdsourced point labels](https://mlsanity.com/paper/2111.03693/) (2021)
- [Towards Cross-Disaster Building Damage Assessment with Graph Convolutional Networks](https://mlsanity.com/paper/2201.10395/) (2022)
- [AB2CD: AI for Building Climate Damage Classification and Detection](https://mlsanity.com/paper/2309.01066/) (2023)

## Abstract (arXiv)

An important step for limiting the negative impact of natural disasters is rapid damage assessment after a disaster occurred. For instance, building damage detection can be automated by applying computer vision techniques to satellite imagery. Such models operate in a multi-domain setting: every disaster is inherently different (new geolocation, unique circumstances), and models must be robust to a shift in distribution between disaster imagery available for training and the images of the new event. Accordingly, estimating real-world performance requires an out-of-domain (OOD) test set. However, building damage detection models have so far been evaluated mostly in the simpler yet unrealistic in-distribution (IID) test setting. Here we argue that future work should focus on the OOD regime instead. We assess OOD performance of two competitive damage detection models and find that existing state-of-the-art models show a substantial generalization gap: their performance drops when evaluated OOD on new disasters not used during training. Moreover, IID performance is not predictive of OOD performance, rendering current benchmarks uninformative about real-world performance. Code and model weights are available at https://github.com/ecker-lab/robust-bdd.