# City Foundation Models for Learning General Purpose Representations from OpenStreetMap

arXiv: https://arxiv.org/abs/2310.00583 · HTML: https://mlsanity.com/paper/2310.00583/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 59 references that are in the map.
- First posted on arXiv 1 Oct 2023.

## Builds on (most cited first, 25 of 25)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation](https://mlsanity.com/paper/2201.12086/) (2022)
- [node2vec: Scalable Feature Learning for Networks](https://mlsanity.com/paper/1607.00653/) (2016)
- [Enriching Word Vectors with Subword Information](https://mlsanity.com/paper/1607.04606/) (2016)
- [A Comprehensive Survey on Transfer Learning](https://mlsanity.com/paper/1911.02685/) (2019)
- [A Comprehensive Survey on Pretrained Foundation Models: A History from BERT to ChatGPT](https://mlsanity.com/paper/2302.09419/) (2023)
- [Multi-Scale Representation Learning for Spatial Feature Distributions using Grid Cells](https://mlsanity.com/paper/2003.00824/) (2020)
- [On the Opportunities and Challenges of Foundation Models for Geospatial Artificial Intelligence](https://mlsanity.com/paper/2304.06798/) (2023)
- [Urban2Vec: Incorporating Street View Imagery and POIs for Multi-Modal Urban Neighborhood Embedding](https://mlsanity.com/paper/2001.11101/) (2020)
- [Region Embedding with Intra and Inter-View Contrastive Learning](https://mlsanity.com/paper/2211.08975/) (2022)
- [Hex2vec -- Context-Aware Embedding H3 Hexagons with OpenStreetMap Tags](https://mlsanity.com/paper/2111.00970/) (2021)
- [SpaBERT: A Pretrained Language Model from Geographic Data for Geo-Entity Representation](https://mlsanity.com/paper/2210.12213/) (2022)
- [Relational Fusion Networks: Graph Convolutional Networks for Road Networks](https://mlsanity.com/paper/2006.09030/) (2020)
- [GeoVectors: A Linked Open Corpus of OpenStreetMap Embeddings on World Scale](https://mlsanity.com/paper/2108.13092/) (2021)
- [Joint Spatio-Textual Reasoning for Answering Tourism Questions](https://mlsanity.com/paper/2009.13613/) (2020)
- [ST-PIL: Spatial-Temporal Periodic Interest Learning for Next Point-of-Interest Recommendation](https://mlsanity.com/paper/2104.02262/) (2021)

## Built on by (most cited first, 21 of 21)

- [TorchSpatial: A Location Encoding Framework and Benchmark for Spatial Representation Learning](https://mlsanity.com/paper/2406.15658/) (2024)
- [Foundation Models for Spatio-Temporal Data Science: A Tutorial and Survey](https://mlsanity.com/paper/2503.13502/) (2025)
- [Towards Urban General Intelligence: A Review and Outlook of Urban Foundation Models](https://mlsanity.com/paper/2402.01749/) (2024)
- [On the Promises and Challenges of Multimodal Foundation Models for Geographical, Environmental, Agricultural, and Urban Planning Applications](https://mlsanity.com/paper/2312.17016/) (2023)
- [Poly2Vec: Polymorphic Fourier-Based Encoding of Geospatial Objects for GeoAI Applications](https://mlsanity.com/paper/2408.14806/) (2024)
- [FlexiReg: Flexible Urban Region Representation Learning](https://mlsanity.com/paper/2503.09128/) (2025)
- [LAMP: A Language Model on the Map](https://mlsanity.com/paper/2403.09059/) (2024)
- [Self-Supervised Representation Learning for Geospatial Objects: A Survey](https://mlsanity.com/paper/2408.12133/) (2024)
- [Urban Region Pre-training and Prompting: A Graph-based Approach](https://mlsanity.com/paper/2408.05920/) (2024)
- [CITYREP: A Unified Benchmark for Urban Representations Across Cities, Tasks, and Modalities](https://mlsanity.com/paper/2605.26036/) (2026)
- [S2Vec: Self-Supervised Geospatial Embeddings for the Built Environment](https://mlsanity.com/paper/2504.16942/) (2025)
- [MoRA: Mobility as the Backbone for Geospatial Representation Learning at Scale](https://mlsanity.com/paper/2506.01297/) (2025)
- [From Points to Places: Towards Human Mobility-Driven Spatiotemporal Foundation Models via Understanding Places](https://mlsanity.com/paper/2506.14570/) (2025)
- [Geography-Aware Large Language Models for Next POI Recommendation](https://mlsanity.com/paper/2505.13526/) (2025)
- [GeoResponder: Towards Building Geospatial LLMs for Time-Critical Disaster Response](https://mlsanity.com/paper/2509.19354/) (2025)
- [UrbanVerse: Learning Urban Region Representation Across Cities and Tasks](https://mlsanity.com/paper/2602.15750/) (2026)
- [Spatial Representation Learning Beyond Pixels: Unifying Raster Data and Vector Semantics for Human-Centric Geospatial Foundation Models](https://mlsanity.com/paper/2606.02374/) (2026)
- [PLMTrajRec: A Scalable and Generalizable Trajectory Recovery Method with Pre-trained Language Models](https://mlsanity.com/paper/2410.14281/) (2024)
- [ToPT: Task-Oriented Prompt Tuning for Urban Region Representation Learning](https://mlsanity.com/paper/2602.01610/) (2026)
- [NARA: Anchor-Conditioned Representation Learning for Heterogeneous Vector Geoentities](https://mlsanity.com/paper/2605.12276/) (2026)
- [MoRAX: Mobility-based Representation Augmentation for Geospatial Foundation Models](https://mlsanity.com/paper/2608.17848/) (2026)

## Abstract (arXiv)

Pre-trained Foundation Models (PFMs) have ushered in a paradigm-shift in Artificial Intelligence, due to their ability to learn general-purpose representations that can be readily employed in a wide range of downstream tasks. While PFMs have been successfully adopted in various fields such as Natural Language Processing and Computer Vision, their capacity in handling geospatial data and answering urban questions remains limited. This can be attributed to the intrinsic heterogeneity of geospatial data, which encompasses different data types, including points, segments and regions, as well as multiple information modalities, such as a spatial position, visual characteristics and textual annotations. The proliferation of Volunteered Geographic Information initiatives, and the ever-increasing availability of open geospatial data sources, like OpenStreetMap, which is freely accessible globally, unveil a promising opportunity to bridge this gap. In this paper, we present CityFM, a self-supervised framework to train a foundation model within a selected geographical area of interest, such as a city. CityFM relies solely on open data from OSM, and produces multimodal representations of entities of different types, incorporating spatial, visual, and textual information. We analyse the entity representations generated using our foundation models from a qualitative perspective, and conduct quantitative experiments on road, building, and region-level downstream tasks. We compare its results to algorithms tailored specifically for the respective applications. In all the experiments, CityFM achieves performance superior to, or on par with, the baselines.