# GeoLM: Empowering Language Models for Geospatially Grounded Language Understanding

arXiv: https://arxiv.org/abs/2310.14478 · HTML: https://mlsanity.com/paper/2310.14478/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning.
- Builds on 28 of its 53 references that are in the map.
- First posted on arXiv 23 Oct 2023.

## Placed in

- LLM reasoning / [Spatial and embodied reasoning](https://mlsanity.com/lr/spatial-and-embodied-reasoning/)

## Builds on (most cited first, 28 of 28)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision](https://mlsanity.com/paper/2102.05918/) (2021)
- [SimCSE: Simple Contrastive Learning of Sentence Embeddings](https://mlsanity.com/paper/2104.08821/) (2021)
- [BioBERT: a pre-trained biomedical language representation model for biomedical text mining](https://mlsanity.com/paper/1901.08746/) (2019)
- [Publicly Available Clinical BERT Embeddings](https://mlsanity.com/paper/1904.03323/) (2019)
- [VL-BERT: Pre-training of Generic Visual-Linguistic Representations](https://mlsanity.com/paper/1908.08530/) (2019)
- [SpanBERT: Improving Pre-training by Representing and Predicting Spans](https://mlsanity.com/paper/1907.10529/) (2019)
- [KEPLER: A Unified Model for Knowledge Embedding and Pre-trained Language Representation](https://mlsanity.com/paper/1911.06136/) (2019)
- [Autoregressive Entity Retrieval](https://mlsanity.com/paper/2010.00904/) (2020)
- [LUKE: Deep Contextualized Entity Representations with Entity-aware Self-attention](https://mlsanity.com/paper/2010.01057/) (2020)
- [Self-Alignment Pretraining for Biomedical Entity Representations](https://mlsanity.com/paper/2010.11784/) (2020)
- [Learning the Best Pooling Strategy for Visual Semantic Embedding](https://mlsanity.com/paper/2011.04305/) (2020)
- [Temporal Adaptation of BERT and Performance on Downstream Document Classification: Insights from Social Media](https://mlsanity.com/paper/2104.08116/) (2021)
- [Code and Named Entity Recognition in StackOverflow](https://mlsanity.com/paper/2005.01634/) (2020)
- [Learning Visual Representation from Modality-Shared Contrastive Language-Image Pre-training](https://mlsanity.com/paper/2207.12661/) (2022)
- [NeuroNER: an easy-to-use program for named-entity recognition based on neural networks](https://mlsanity.com/paper/1705.05487/) (2017)
- [SpaBERT: A Pretrained Language Model from Geographic Data for Geo-Entity Representation](https://mlsanity.com/paper/2210.12213/) (2022)
- [A Pragmatic Guide to Geoparsing Evaluation](https://mlsanity.com/paper/1810.12368/) (2018)
- [Location reference recognition from texts: A survey and comparison](https://mlsanity.com/paper/2207.01683/) (2022)
- [JuriBERT: A Masked-Language Model Adaptation for French Legal Text](https://mlsanity.com/paper/2110.01485/) (2021)
- [Enhancing spatial and textual analysis with EUPEG: an extensible and unified platform for evaluating geoparsers](https://mlsanity.com/paper/2007.04524/) (2020)
- [BERTweetFR : Domain Adaptation of Pre-Trained Language Models for French Tweets](https://mlsanity.com/paper/2109.10234/) (2021)
- [Mordecai 3: A Neural Geoparser and Event Geocoder](https://mlsanity.com/paper/2303.13675/) (2023)
- [How can voting mechanisms improve the robustness and generalizability of toponym disambiguation?](https://mlsanity.com/paper/2209.08286/) (2022)

## Built on by (most cited first, 30 of 37)

- [A Comprehensive Survey of Scientific Large Language Models and Their Applications in Scientific Discovery](https://mlsanity.com/paper/2406.10833/) (2024)
- [Urban Generative Intelligence (UGI): A Foundational Platform for Agents in Embodied City Environment](https://mlsanity.com/paper/2312.11813/) (2023)
- [CityGPT: Empowering Urban Spatial Cognition of Large Language Models](https://mlsanity.com/paper/2406.13948/) (2024)
- [Foundation Models for Spatio-Temporal Data Science: A Tutorial and Survey](https://mlsanity.com/paper/2503.13502/) (2025)
- [Towards Urban General Intelligence: A Review and Outlook of Urban Foundation Models](https://mlsanity.com/paper/2402.01749/) (2024)
- [UrbanKGent: A Unified Large Language Model Agent Framework for Urban Knowledge Graph Construction](https://mlsanity.com/paper/2402.06861/) (2024)
- [MapQA: Open-domain Geospatial Question Answering on Map Data](https://mlsanity.com/paper/2503.07871/) (2025)
- [AI Sees Your Location, But With A Bias Toward The Wealthy World](https://mlsanity.com/paper/2502.11163/) (2025)
- [GeoLocator: a location-integrated large multimodal model for inferring geo-privacy](https://mlsanity.com/paper/2311.13018/) (2023)
- [Spatial-RAG: Spatial Retrieval Augmented Generation for Real-World Geospatial Reasoning Questions](https://mlsanity.com/paper/2502.18470/) (2025)
- [Evaluating the Ability of Large Language Models to Reason about Cardinal Directions](https://mlsanity.com/paper/2406.16528/) (2024)
- [GeoReasoner: Reasoning On Geospatially Grounded Context For Natural Language Understanding](https://mlsanity.com/paper/2408.11366/) (2024)
- [Unraveling Spatio-Temporal Foundation Models via the Pipeline Lens: A Comprehensive Review](https://mlsanity.com/paper/2506.01364/) (2025)
- [Self-Supervised Representation Learning for Geospatial Objects: A Survey](https://mlsanity.com/paper/2408.12133/) (2024)
- [Evaluation of Geographical Distortions in Language Models](https://mlsanity.com/paper/2404.17401/) (2024)
- [Where on Earth Do Users Say They Are?: Geo-Entity Linking for Noisy Multilingual User Input](https://mlsanity.com/paper/2404.18784/) (2024)
- [MoRA: Mobility as the Backbone for Geospatial Representation Learning at Scale](https://mlsanity.com/paper/2506.01297/) (2025)
- [Mitigating Geospatial Knowledge Hallucination in Large Language Models: Benchmarking and Dynamic Factuality Aligning](https://mlsanity.com/paper/2507.19586/) (2025)
- [Mobility-Embedded POIs: Learning What A Place Is and How It Is Used from Human Movement](https://mlsanity.com/paper/2601.21149/) (2026)
- [DisasterBench: Benchmarking LLM Planning under Typed Tool Interface Constraints](https://mlsanity.com/paper/2605.27957/) (2026)
- [Exploring Language Model Generalization in Low-Resource Extractive QA](https://mlsanity.com/paper/2409.18446/) (2024)
- [Comparative Performance of Advanced NLP Models and LLMs in Multilingual Geo-Entity Detection](https://mlsanity.com/paper/2412.20414/) (2024)
- [Urban Computing in the Era of Large Language Models](https://mlsanity.com/paper/2504.02009/) (2025)
- [A Vision for Geo-Temporal Deep Research Systems: Towards Comprehensive, Transparent, and Reproducible Geo-Temporal Information Synthesis](https://mlsanity.com/paper/2506.14345/) (2025)
- [DIGMAPPER: A Modular System for Automated Geologic Map Digitization](https://mlsanity.com/paper/2506.16006/) (2025)
- [Graph-Structured Trajectory Extraction from Travelogues](https://mlsanity.com/paper/2410.16633/) (2024)
- [Evaluating the Ability of Large Language Models to Reason about Cardinal Directions, Revisited](https://mlsanity.com/paper/2507.12059/) (2025)
- [GeoResponder: Towards Building Geospatial LLMs for Time-Critical Disaster Response](https://mlsanity.com/paper/2509.19354/) (2025)
- [Knowledge Graph-Guided Multi-Agent Distillation for Reliable Industrial Question Answering with Datasets](https://mlsanity.com/paper/2510.06240/) (2025)
- [Detecting Legend Items on Historical Maps Using GPT-4o with In-Context Learning](https://mlsanity.com/paper/2510.08385/) (2025)

## Abstract (arXiv)

Humans subconsciously engage in geospatial reasoning when reading articles. We recognize place names and their spatial relations in text and mentally associate them with their physical locations on Earth. Although pretrained language models can mimic this cognitive process using linguistic context, they do not utilize valuable geospatial information in large, widely available geographical databases, e.g., OpenStreetMap. This paper introduces GeoLM, a geospatially grounded language model that enhances the understanding of geo-entities in natural language. GeoLM leverages geo-entity mentions as anchors to connect linguistic information in text corpora with geospatial information extracted from geographical databases. GeoLM connects the two types of context through contrastive learning and masked language modeling. It also incorporates a spatial coordinate embedding mechanism to encode distance and direction relations to capture geospatial context. In the experiment, we demonstrate that GeoLM exhibits promising capabilities in supporting toponym recognition, toponym linking, relation extraction, and geo-entity typing, which bridge the gap between natural language processing and geospatial sciences. The code is publicly available at https://github.com/knowledge-computing/geolm.