# OpenFACADES: An Open Framework for Architectural Caption and Attribute Data Enrichment via Street View Imagery

arXiv: https://arxiv.org/abs/2504.02866 · HTML: https://mlsanity.com/paper/2504.02866/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 82 references that are in the map.
- First posted on arXiv 1 Apr 2025.

## Builds on (most cited first, 15 of 15)

- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [RSGPT: A Remote Sensing Vision Language Model and Benchmark](https://mlsanity.com/paper/2307.15266/) (2023)
- [OSMnx: New Methods for Acquiring, Constructing, Analyzing, and Visualizing Complex Street Networks](https://mlsanity.com/paper/1611.01890/) (2016)
- [Vision-Language Models in Remote Sensing: Current Progress and Future Trends](https://mlsanity.com/paper/2305.05726/) (2023)
- [Building Instance Classification Using Street View Images](https://mlsanity.com/paper/1802.09026/) (2018)
- [Cross-View Geolocalization and Disaster Mapping with Street-View and VHR Satellite Imagery: A Case Study of Hurricane IAN](https://mlsanity.com/paper/2408.06761/) (2024)
- [Coverage and Bias of Street View Imagery in Mapping the Urban Environment](https://mlsanity.com/paper/2409.15386/) (2024)
- [To use or not to use proprietary street view images in (health and place) research? That is the question](https://mlsanity.com/paper/2402.11504/) (2024)
- [Roofpedia: Automatic mapping of green and solar roofs for an open roofscape registry and evaluation of urban sustainability](https://mlsanity.com/paper/2012.14349/) (2020)
- [A citizen science toolkit to collect human perceptions of urban environments using open street view images](https://mlsanity.com/paper/2403.00174/) (2024)
- [Estimating building energy efficiency from street view imagery, aerial imagery, and land surface temperature data](https://mlsanity.com/paper/2206.02270/) (2022)
- [Zero-shot Building Age Classification from Facade Image Using GPT-4](https://mlsanity.com/paper/2404.09921/) (2024)
- [StreetSurfaceVis: a dataset of crowdsourced street-level imagery annotated by road surface type and quality](https://mlsanity.com/paper/2407.21454/) (2024)
- [ZenSVI: An Open-Source Software for the Integrated Acquisition, Processing and Analysis of Street View Imagery Towards Scalable Urban Science](https://mlsanity.com/paper/2412.18641/) (2024)
- [UB-FineNet: Urban Building Fine-grained Classification Network for Open-access Satellite Images](https://mlsanity.com/paper/2403.02132/) (2024)

## Built on by (most cited first, 8 of 8)

- [Recov-Vision: Linking Street View Imagery and Vision-Language Models for Post-Disaster Recovery](https://mlsanity.com/paper/2509.20628/) (2025)
- [Automated Building Heritage Assessment Using Street-Level Imagery](https://mlsanity.com/paper/2508.11486/) (2025)
- [Reasoning Is All You Need for Urban Planning AI](https://mlsanity.com/paper/2511.05375/) (2025)
- [Using street view images and visual LLMs to predict heritage values for governance support: Risks, ethics, and policy implications](https://mlsanity.com/paper/2601.06056/) (2025)
- [Leveraging Multimodal LLMs for Built Environment and Housing Attribute Assessment from Street-View Imagery](https://mlsanity.com/paper/2604.21102/) (2026)
- [City landscape in sight: A crowdsourced framework for unlocking urban-scale window view perceptions from real estate imagery](https://mlsanity.com/paper/2606.15198/) (2026)
- [Can Urban Blight Be Accessed with Vision-language Models: A Case Study in Detroit](https://mlsanity.com/paper/2608.01753/) (2026)
- [Seeing the City or Recognizing the Place? What Street-View Imagery Adds Beyond Existing Urban Data in VLM-Based Urban Sensing](https://mlsanity.com/paper/2610.00031/) (2026)

## Abstract (arXiv)

Building properties, such as height, usage, and material, play a crucial role in spatial data infrastructures, supporting various urban applications. Despite their importance, comprehensive building attribute data remain scarce in many urban areas. Recent advances have enabled the extraction of objective building attributes using remote sensing and street-level imagery. However, establishing a pipeline that integrates diverse open datasets, acquires holistic building imagery, and infers comprehensive building attributes at scale remains a significant challenge. Among the first, this study bridges the gaps by introducing OpenFACADES, an open framework that leverages multimodal crowdsourced data to enrich building profiles with both objective attributes and semantic descriptors through multimodal large language models. First, we integrate street-level image metadata from Mapillary with OpenStreetMap geometries via isovist analysis, identifying images that provide suitable vantage points for observing target buildings. Second, we automate the detection of building facades in panoramic imagery and tailor a reprojection approach to convert objects into holistic perspective views that approximate real-world observation. Third, we introduce an innovative approach that harnesses and investigates the capabilities of open-source large vision-language models (VLMs) for multi-attribute prediction and open-vocabulary captioning in building-level analytics, leveraging a globally sourced dataset of 31,180 labeled images from seven cities. Evaluation shows that fine-tuned VLM excel in multi-attribute inference, outperforming single-attribute computer vision models and zero-shot ChatGPT-4o. Further experiments confirm its superior generalization and robustness across culturally distinct region and varying image conditions.