# RescueNet: Joint Building Segmentation and Damage Assessment from Satellite Imagery

arXiv: https://arxiv.org/abs/2004.07312 · HTML: https://mlsanity.com/paper/2004.07312/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Apr 2020.

## Built on by (most cited first, 24 of 24)

- [Self-Supervised Material and Texture Representation Learning for Remote Sensing Tasks](https://mlsanity.com/paper/2112.01715/) (2021)
- [BRIGHT: A globally distributed multimodal building damage assessment dataset with very-high-resolution for all-weather disaster response](https://mlsanity.com/paper/2501.06019/) (2025)
- [Large-scale Building Damage Assessment using a Novel Hierarchical Transformer Architecture on Satellite Images](https://mlsanity.com/paper/2208.02205/) (2022)
- [BDANet: Multiscale Convolutional Neural Network with Cross-directional Attention for Building Damage Assessment from Satellite Images](https://mlsanity.com/paper/2105.07364/) (2021)
- [Generalizable Disaster Damage Assessment via Change Detection with Vision Foundation Model](https://mlsanity.com/paper/2406.08020/) (2024)
- [Rapid building damage assessment workflow: An implementation for the 2023 Rolling Fork, Mississippi tornado event](https://mlsanity.com/paper/2306.12589/) (2023)
- [DamageCAT: A Deep Learning Transformer Framework for Typology-Based Post-Disaster Building Damage Categorization](https://mlsanity.com/paper/2504.11637/) (2025)
- [Temporal Cluster Matching for Change Detection of Structures from Satellite Imagery](https://mlsanity.com/paper/2103.09787/) (2021)
- [Attention Based Semantic Segmentation on UAV Dataset for Natural Disaster Damage Assessment](https://mlsanity.com/paper/2105.14540/) (2021)
- [Self-Supervised Learning for Building Damage Assessment from Large-scale xBD Satellite Imagery Benchmark Datasets](https://mlsanity.com/paper/2205.15688/) (2022)
- [Robust Disaster Assessment from Aerial Imagery Using Text-to-Image Synthetic Data](https://mlsanity.com/paper/2405.13779/) (2024)
- [Learning Efficient Unsupervised Satellite Image-based Building Damage Detection](https://mlsanity.com/paper/2312.01576/) (2023)
- [Building Damage Mapping with Self-PositiveUnlabeled Learning](https://mlsanity.com/paper/2111.02586/) (2021)
- [Disaster mapping from satellites: damage detection with crowdsourced point labels](https://mlsanity.com/paper/2111.03693/) (2021)
- [GaLeNet: Multimodal Learning for Disaster Prediction, Management and Relief](https://mlsanity.com/paper/2206.09242/) (2022)
- [Multi-step feature fusion for natural disaster damage assessment on satellite images](https://mlsanity.com/paper/2410.21901/) (2024)
- [The Potential of Copernicus Satellites for Disaster Response: Retrieving Building Damage from Sentinel-1 and Sentinel-2](https://mlsanity.com/paper/2511.05461/) (2025)
- [Damage Estimation and Localization from Sparse Aerial Imagery](https://mlsanity.com/paper/2111.03708/) (2021)
- [Interpretability in Convolutional Neural Networks for Building Damage Classification in Satellite Imagery](https://mlsanity.com/paper/2201.10523/) (2022)
- [Building Height Prediction with Instance Segmentation](https://mlsanity.com/paper/2212.09277/) (2022)
- [AI and Remote Sensing for Resilient and Sustainable Built Environments: A Review of Current Methods, Open Data and Future Directions](https://mlsanity.com/paper/2507.01547/) (2025)
- [Architectural Insights for Post-Tornado Damage Recognition](https://mlsanity.com/paper/2602.14523/) (2026)
- [HASTE: A Platform for Rapid Post-Disaster Building Damage Assessment](https://mlsanity.com/paper/2607.11838/) (2026)
- [Obshazard-bench: Benchmarking Multimodal Foundation Models for Real-Time Disaster Intelligence from Raw Earth Observation Streams](https://mlsanity.com/paper/2608.00012/) (2026)

## Abstract (arXiv)

Accurate and fine-grained information about the extent of damage to buildings is essential for directing Humanitarian Aid and Disaster Response (HADR) operations in the immediate aftermath of any natural calamity. In recent years, satellite and UAV (drone) imagery has been used for this purpose, sometimes aided by computer vision algorithms. Existing Computer Vision approaches for building damage assessment typically rely on a two stage approach, consisting of building detection using an object detection model, followed by damage assessment through classification of the detected building tiles. These multi-stage methods are not end-to-end trainable, and suffer from poor overall results. We propose RescueNet, a unified model that can simultaneously segment buildings and assess the damage levels to individual buildings and can be trained end-toend. In order to to model the composite nature of this problem, we propose a novel localization aware loss function, which consists of a Binary Cross Entropy loss for building segmentation, and a foreground only selective Categorical Cross-Entropy loss for damage classification, and show significant improvement over the widely used Cross-Entropy loss. RescueNet is tested on the large scale and diverse xBD dataset and achieves significantly better building segmentation and damage classification performance than previous methods and achieves generalization across varied geographical regions and disaster types.