# Knowledge-infused Contrastive Learning for Urban Imagery-based Socioeconomic Prediction

arXiv: https://arxiv.org/abs/2302.13094 · HTML: https://mlsanity.com/paper/2302.13094/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 60 references that are in the map.
- First posted on arXiv 25 Feb 2023.

## Builds on (most cited first, 23 of 23)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Self-supervised Learning: Generative or Contrastive](https://mlsanity.com/paper/2006.08218/) (2020)
- [Self-supervised Visual Feature Learning with Deep Neural Networks: A Survey](https://mlsanity.com/paper/1902.06162/) (2019)
- [Contrastive Learning of Medical Visual Representations from Paired Images and Text](https://mlsanity.com/paper/2010.00747/) (2020)
- [Knowledge Graphs](https://mlsanity.com/paper/2003.02320/) (2020)
- [Composition-based Multi-Relational Graph Convolutional Networks](https://mlsanity.com/paper/1911.03082/) (2019)
- [TuckER: Tensor Factorization for Knowledge Graph Completion](https://mlsanity.com/paper/1901.09590/) (2019)
- [Self-supervised Learning in Remote Sensing: A Review](https://mlsanity.com/paper/2206.13188/) (2022)
- [Tile2Vec: Unsupervised representation learning for spatially distributed data](https://mlsanity.com/paper/1805.02855/) (2018)
- [A Generalizable and Accessible Approach to Machine Learning with Global Satellite Imagery](https://mlsanity.com/paper/2010.08168/) (2020)
- [Using satellite imagery to understand and promote sustainable development](https://mlsanity.com/paper/2010.06988/) (2020)
- [Using Deep Learning and Google Street View to Estimate the Demographic Makeup of the US](https://mlsanity.com/paper/1702.06683/) (2017)
- [Urban2Vec: Incorporating Street View Imagery and POIs for Multi-Modal Urban Neighborhood Embedding](https://mlsanity.com/paper/2001.11101/) (2020)
- [Multi-Graph Fusion Networks for Urban Region Embedding](https://mlsanity.com/paper/2201.09760/) (2022)
- [Spatio-Temporal Urban Knowledge Graph Enabled Mobility Prediction](https://mlsanity.com/paper/2111.03465/) (2021)
- [Take a Look Around: Using Street View and Satellite Images to Estimate House Prices](https://mlsanity.com/paper/1807.07155/) (2018)
- [Lightweight and Robust Representation of Economic Scales from Satellite Imagery](https://mlsanity.com/paper/1912.08197/) (2019)
- [Learning Economic Indicators by Aggregating Multi-Level Geospatial Information](https://mlsanity.com/paper/2205.01472/) (2022)
- [Knowledge-driven Site Selection via Urban Knowledge Graph](https://mlsanity.com/paper/2111.00787/) (2021)
- [A gridded establishment dataset as a proxy for economic activity in China](https://mlsanity.com/paper/2011.00299/) (2020)

## Built on by (most cited first, 23 of 23)

- [UrbanCLIP: Learning Text-enhanced Urban Region Profiling with Contrastive Language-Image Pretraining from the Web](https://mlsanity.com/paper/2310.18340/) (2023)
- [Deep Learning for Cross-Domain Data Fusion in Urban Computing: Taxonomy, Advances, and Outlook](https://mlsanity.com/paper/2402.19348/) (2024)
- [Towards Urban General Intelligence: A Review and Outlook of Urban Foundation Models](https://mlsanity.com/paper/2402.01749/) (2024)
- [UUKG: Unified Urban Knowledge Graph Dataset for Urban Spatiotemporal Prediction](https://mlsanity.com/paper/2306.11443/) (2023)
- [UrbanVLP: Multi-Granularity Vision-Language Pretraining for Urban Socioeconomic Indicator Prediction](https://mlsanity.com/paper/2403.16831/) (2024)
- [MuseCL: Predicting Urban Socioeconomic Indicators via Multi-Semantic Contrastive Learning](https://mlsanity.com/paper/2407.09523/) (2024)
- [Geolocation Representation from Large Language Models are Generic Enhancers for Spatio-Temporal Learning](https://mlsanity.com/paper/2408.12116/) (2024)
- [Urban Region Embedding via Multi-View Contrastive Prediction](https://mlsanity.com/paper/2312.09681/) (2023)
- [Urban Region Pre-training and Prompting: A Graph-based Approach](https://mlsanity.com/paper/2408.05920/) (2024)
- [Spatial Heterophily Aware Graph Neural Networks](https://mlsanity.com/paper/2306.12139/) (2023)
- [CityLens: Evaluating Large Vision-Language Models for Urban Socioeconomic Sensing](https://mlsanity.com/paper/2506.00530/) (2025)
- [A Satellite Imagery Dataset for Long-Term Sustainable Development in United States Cities](https://mlsanity.com/paper/2308.00465/) (2023)
- [USTBench: Benchmarking and Dissecting Spatiotemporal Reasoning of LLMs as Urban Agents](https://mlsanity.com/paper/2505.17572/) (2025)
- [CureGraph: Contrastive Multi-Modal Graph Representation Learning for Urban Living Circle Health Profiling and Prediction](https://mlsanity.com/paper/2501.07157/) (2025)
- [Devil in the Landscapes: Inferring Epidemic Exposure Risks from Street View Imagery](https://mlsanity.com/paper/2311.09240/) (2023)
- [UrbanMoE: A Sparse Multi-Modal Mixture-of-Experts Framework for Multi-Task Urban Region Profiling](https://mlsanity.com/paper/2601.22746/) (2026)
- [Enhancing Ride-Hailing Forecasting at DiDi with Multi-View Geospatial Representation Learning from the Web](https://mlsanity.com/paper/2602.10502/) (2026)
- [Self-supervised learning unveils change in urban housing from street-level images](https://mlsanity.com/paper/2309.11354/) (2023)
- [Multi-Agent Collaborative Reasoning with Tool-Augmented Evidence for Urban Region Profiling](https://mlsanity.com/paper/2607.13558/) (2026)
- [CoST: Semantic-Aware Urban Understanding via Spatial-Temporal Alignment](https://mlsanity.com/paper/2608.21041/) (2026)
- [Learning Street View Representations with Spatiotemporal Contrast](https://mlsanity.com/paper/2502.04638/) (2025)
- [UrbanWell: Benchmarking Multimodal Large Language Models for Spatio-Temporal Urban Wellbeing Analytics](https://mlsanity.com/paper/2606.15890/) (2026)
- [Synergistic Fusion of Topological Structure and Temporal Semantics of Mobility for Urban Region Embedding](https://mlsanity.com/paper/2609.08268/) (2026)

## Abstract (arXiv)

Monitoring sustainable development goals requires accurate and timely socioeconomic statistics, while ubiquitous and frequently-updated urban imagery in web like satellite/street view images has emerged as an important source for socioeconomic prediction. Especially, recent studies turn to self-supervised contrastive learning with manually designed similarity metrics for urban imagery representation learning and further socioeconomic prediction, which however suffers from effectiveness and robustness issues. To address such issues, in this paper, we propose a Knowledge-infused Contrastive Learning (KnowCL) model for urban imagery-based socioeconomic prediction. Specifically, we firstly introduce knowledge graph (KG) to effectively model the urban knowledge in spatiality, mobility, etc., and then build neural network based encoders to learn representations of an urban image in associated semantic and visual spaces, respectively. Finally, we design a cross-modality based contrastive learning framework with a novel image-KG contrastive loss, which maximizes the mutual information between semantic and visual representations for knowledge infusion. Extensive experiments of applying the learnt visual representations for socioeconomic prediction on three datasets demonstrate the superior performance of KnowCL with over 30% improvements on R2 compared with baselines. Especially, our proposed KnowCL model can apply to both satellite and street imagery with both effectiveness and transferability achieved, which provides insights into urban imagery-based socioeconomic prediction.