# Estimating building energy efficiency from street view imagery, aerial imagery, and land surface temperature data

arXiv: https://arxiv.org/abs/2206.02270 · HTML: https://mlsanity.com/paper/2206.02270/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 48 references that are in the map.
- First posted on arXiv 5 Jun 2022.

## Builds on (most cited first, 10 of 10)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Captum: A unified and generic model interpretability library for PyTorch](https://mlsanity.com/paper/2009.07896/) (2020)
- [Probabilistic Model-Agnostic Meta-Learning](https://mlsanity.com/paper/1806.02817/) (2018)
- [Using Deep Learning and Google Street View to Estimate the Demographic Makeup of the US](https://mlsanity.com/paper/1702.06683/) (2017)
- [An Enriched Automated PV Registry: Combining Image Recognition and 3D Building Data](https://mlsanity.com/paper/2012.03690/) (2020)

## Built on by (most cited first, 8 of 8)

- [PIGEON: Predicting Image Geolocations](https://mlsanity.com/paper/2307.05845/) (2023)
- [Coverage and Bias of Street View Imagery in Mapping the Urban Environment](https://mlsanity.com/paper/2409.15386/) (2024)
- [OpenFACADES: An Open Framework for Architectural Caption and Attribute Data Enrichment via Street View Imagery](https://mlsanity.com/paper/2504.02866/) (2025)
- [It is not always greener on the other side: Greenery perception across demographics and personalities in multiple cities](https://mlsanity.com/paper/2512.17186/) (2025)
- [SENSE: Satellite-based ENergy Synthesis for Sustainable Environment](https://mlsanity.com/paper/2605.18101/) (2026)
- [Points for Energy Renovation (PointER): A LiDAR-Derived Point Cloud Dataset of One Million English Buildings Linked to Energy Characteristics](https://mlsanity.com/paper/2306.16020/) (2023)
- [ConBatch-BAL: Batch Bayesian Active Learning under Budget Constraints](https://mlsanity.com/paper/2507.04929/) (2025)
- [Predicting Household Water Consumption Using Satellite and Street View Images in Two Indian Cities](https://mlsanity.com/paper/2510.26957/) (2025)

## Abstract (arXiv)

Current methods to determine the energy efficiency of buildings require on-site visits of certified energy auditors which makes the process slow, costly, and geographically incomplete. To accelerate the identification of promising retrofit targets on a large scale, we propose to estimate building energy efficiency from widely available and remotely sensed data sources only, namely street view, aerial view, footprint, and satellite-borne land surface temperature (LST) data. After collecting data for almost 40,000 buildings in the United Kingdom, we combine these data sources by training multiple end-to-end deep learning models with the objective to classify buildings as energy efficient (EU rating A-D) or inefficient (EU rating E-G). After evaluating the trained models quantitatively as well as qualitatively, we extend our analysis by studying the predictive power of each data source in an ablation study. We find that the end-to-end deep learning model trained on all four data sources achieves a macro-averaged F1 score of 64.64% and outperforms the k-NN and SVM-based baseline models by 14.13 to 12.02 percentage points, respectively. Thus, this work shows the potential and complementary nature of remotely sensed data in predicting energy efficiency and opens up new opportunities for future work to integrate additional data sources.