# Urban2Vec: Incorporating Street View Imagery and POIs for Multi-Modal Urban Neighborhood Embedding

arXiv: https://arxiv.org/abs/2001.11101 · HTML: https://mlsanity.com/paper/2001.11101/

## Key facts

- Cited by 44 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Jan 2020.

## Built on by (most cited first, 30 of 44)

- [UrbanCLIP: Learning Text-enhanced Urban Region Profiling with Contrastive Language-Image Pretraining from the Web](https://mlsanity.com/paper/2310.18340/) (2023)
- [Hex2vec -- Context-Aware Embedding H3 Hexagons with OpenStreetMap Tags](https://mlsanity.com/paper/2111.00970/) (2021)
- [Knowledge-infused Contrastive Learning for Urban Imagery-based Socioeconomic Prediction](https://mlsanity.com/paper/2302.13094/) (2023)
- [Towards Urban General Intelligence: A Review and Outlook of Urban Foundation Models](https://mlsanity.com/paper/2402.01749/) (2024)
- [City Foundation Models for Learning General Purpose Representations from OpenStreetMap](https://mlsanity.com/paper/2310.00583/) (2023)
- [UrbanVLP: Multi-Granularity Vision-Language Pretraining for Urban Socioeconomic Indicator Prediction](https://mlsanity.com/paper/2403.16831/) (2024)
- [MuseCL: Predicting Urban Socioeconomic Indicators via Multi-Semantic Contrastive Learning](https://mlsanity.com/paper/2407.09523/) (2024)
- [Geolocation Representation from Large Language Models are Generic Enhancers for Spatio-Temporal Learning](https://mlsanity.com/paper/2408.12116/) (2024)
- [MM-Path: Multi-modal, Multi-granularity Path Representation Learning -- Extended Version](https://mlsanity.com/paper/2411.18428/) (2024)
- [Urban Region Embedding via Multi-View Contrastive Prediction](https://mlsanity.com/paper/2312.09681/) (2023)
- [FlexiReg: Flexible Urban Region Representation Learning](https://mlsanity.com/paper/2503.09128/) (2025)
- [Urban Region Representation Learning with Attentive Fusion](https://mlsanity.com/paper/2312.04606/) (2023)
- [Combining Discrete Choice Models and Neural Networks through Embeddings: Formulation, Interpretability and Performance](https://mlsanity.com/paper/2109.12042/) (2021)
- [Learning Neighborhood Representation from Multi-Modal Multi-Graph: Image, Text, Mobility Graph and Beyond](https://mlsanity.com/paper/2105.02489/) (2021)
- [Self-Supervised Representation Learning for Geospatial Objects: A Survey](https://mlsanity.com/paper/2408.12133/) (2024)
- [CityPulse: Fine-Grained Assessment of Urban Change with Street View Time Series](https://mlsanity.com/paper/2401.01107/) (2024)
- [Effective Urban Region Representation Learning Using Heterogeneous Urban Graph Attention Network (HUGAT)](https://mlsanity.com/paper/2202.09021/) (2022)
- [Space-aware Socioeconomic Indicator Inference with Heterogeneous Graphs](https://mlsanity.com/paper/2405.14135/) (2024)
- [Anchors Aweigh! Sail for Optimal Unified Multi-Modal Representations](https://mlsanity.com/paper/2410.02086/) (2024)
- [Using Multiple Input Modalities Can Improve Data-Efficiency and O.O.D. Generalization for ML with Satellite Imagery](https://mlsanity.com/paper/2507.13385/) (2025)
- [CITYREP: A Unified Benchmark for Urban Representations Across Cities, Tasks, and Modalities](https://mlsanity.com/paper/2605.26036/) (2026)
- [Geographic ratemaking with spatial embeddings](https://mlsanity.com/paper/2104.12852/) (2021)
- [Detecting Neighborhood Gentrification at Scale via Street-level Visual Data](https://mlsanity.com/paper/2301.01842/) (2023)
- [Energy-Based Models for Cross-Modal Localization using Convolutional Transformers](https://mlsanity.com/paper/2306.04021/) (2023)
- [CureGraph: Contrastive Multi-Modal Graph Representation Learning for Urban Living Circle Health Profiling and Prediction](https://mlsanity.com/paper/2501.07157/) (2025)
- [Urban-R1: Reinforced MLLMs Mitigate Geospatial Biases for Urban General Intelligence](https://mlsanity.com/paper/2510.16555/) (2025)
- [Neural Embeddings of Urban Big Data Reveal Emergent Structures in Cities](https://mlsanity.com/paper/2110.12371/) (2021)
- [Devil in the Landscapes: Inferring Epidemic Exposure Risks from Street View Imagery](https://mlsanity.com/paper/2311.09240/) (2023)
- [GeoDecoder: Empowering Multimodal Map Understanding](https://mlsanity.com/paper/2401.15118/) (2024)
- [Self-supervised learning unveils change in urban housing from street-level images](https://mlsanity.com/paper/2309.11354/) (2023)

## Abstract (arXiv)

Understanding intrinsic patterns and predicting spatiotemporal characteristics of cities require a comprehensive representation of urban neighborhoods. Existing works relied on either inter- or intra-region connectivities to generate neighborhood representations but failed to fully utilize the informative yet heterogeneous data within neighborhoods. In this work, we propose Urban2Vec, an unsupervised multi-modal framework which incorporates both street view imagery and point-of-interest (POI) data to learn neighborhood embeddings. Specifically, we use a convolutional neural network to extract visual features from street view images while preserving geospatial similarity. Furthermore, we model each POI as a bag-of-words containing its category, rating, and review information. Analog to document embedding in natural language processing, we establish the semantic similarity between neighborhood ("document") and the words from its surrounding POIs in the vector space. By jointly encoding visual, textual, and geospatial information into the neighborhood representation, Urban2Vec can achieve performances better than baseline models and comparable to fully-supervised methods in downstream prediction tasks. Extensive experiments on three U.S. metropolitan areas also demonstrate the model interpretability, generalization capability, and its value in neighborhood similarity analysis.