# CitySurfaces: City-Scale Semantic Segmentation of Sidewalk Materials

arXiv: https://arxiv.org/abs/2201.02260 · HTML: https://mlsanity.com/paper/2201.02260/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 67 references that are in the map.
- First posted on arXiv 6 Jan 2022.

## Builds on (most cited first, 15 of 15)

- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [Deep High-Resolution Representation Learning for Visual Recognition](https://mlsanity.com/paper/1908.07919/) (2019)
- [Segmentation Transformer: Object-Contextual Representations for Semantic Segmentation](https://mlsanity.com/paper/1909.11065/) (2019)
- [High-Resolution Representations for Labeling Pixels and Regions](https://mlsanity.com/paper/1904.04514/) (2019)
- [Hierarchical Multi-Scale Attention for Semantic Segmentation](https://mlsanity.com/paper/2005.10821/) (2020)
- [Attention to Scale: Scale-aware Semantic Image Segmentation](https://mlsanity.com/paper/1511.03339/) (2015)
- [Improving Semantic Segmentation via Video Propagation and Label Relaxation](https://mlsanity.com/paper/1812.01593/) (2018)
- [OSMnx: New Methods for Acquiring, Constructing, Analyzing, and Visualizing Complex Street Networks](https://mlsanity.com/paper/1611.01890/) (2016)
- [Region Mutual Information Loss for Semantic Segmentation](https://mlsanity.com/paper/1910.12037/) (2019)
- [A Method for Stopping Active Learning Based on Stabilizing Predictions and the Need for User-Adjustable Stopping](https://mlsanity.com/paper/1409.5165/) (2014)
- [Urban Mosaic: Visual Exploration of Streetscapes Using Large-Scale Image Data](https://mlsanity.com/paper/2008.13321/) (2020)
- [Shadow Accrual Maps: Efficient Accumulation of City-Scale Shadows Over Time](https://mlsanity.com/paper/1907.04435/) (2019)
- [Message Passing Adaptive Resonance Theory for Online Active Semi-supervised Learning](https://mlsanity.com/paper/2012.01227/) (2020)
- [Recognizing Textures with Mobile Cameras for Pedestrian Safety Applications](https://mlsanity.com/paper/1711.00558/) (2017)

## Built on by (most cited first, 8 of 8)

- [Streetscape Analysis with Generative AI (SAGAI): Vision-Language Assessment and Mapping of Urban Scenes](https://mlsanity.com/paper/2504.16538/) (2025)
- [StreetReaderAI: Making Street View Accessible Using Context-Aware Multimodal AI](https://mlsanity.com/paper/2508.08524/) (2025)
- [Towards Global-Scale Crowd+AI Techniques to Map and Assess Sidewalks for People with Disabilities](https://mlsanity.com/paper/2206.13677/) (2022)
- [ELSA: Evaluating Localization of Social Activities in Urban Streets using Open-Vocabulary Detection](https://mlsanity.com/paper/2406.01551/) (2024)
- [SideSeeing: A multimodal dataset and collection of tools for sidewalk assessment](https://mlsanity.com/paper/2407.06464/) (2024)
- [Remote sensing colour image semantic segmentation of trails created by large herbivorous Mammals](https://mlsanity.com/paper/2504.12121/) (2025)
- ["Does the cafe entrance look accessible? Where is the door?" Towards Geospatial AI Agents for Visual Inquiries](https://mlsanity.com/paper/2508.15752/) (2025)
- [From Street View Imagery to Street Quality Indicators: Vision Language Inference for the Suburban 15-minute City](https://mlsanity.com/paper/2608.20026/) (2026)

## Abstract (arXiv)

While designing sustainable and resilient urban built environment is increasingly promoted around the world, significant data gaps have made research on pressing sustainability issues challenging to carry out. Pavements are known to have strong economic and environmental impacts; however, most cities lack a spatial catalog of their surfaces due to the cost-prohibitive and time-consuming nature of data collection. Recent advancements in computer vision, together with the availability of street-level images, provide new opportunities for cities to extract large-scale built environment data with lower implementation costs and higher accuracy. In this paper, we propose CitySurfaces, an active learning-based framework that leverages computer vision techniques for classifying sidewalk materials using widely available street-level images. We trained the framework on images from New York City and Boston and the evaluation results show a 90.5% mIoU score. Furthermore, we evaluated the framework using images from six different cities, demonstrating that it can be applied to regions with distinct urban fabrics, even outside the domain of the training data. CitySurfaces can provide researchers and city agencies with a low-cost, accurate, and extensible method to collect sidewalk material data which plays a critical role in addressing major sustainability issues, including climate change and surface water management.