mlsanityData to 7 Oct 2026

Are Large Language Models Geospatially Knowledgeable?

arXiv 2310.13002 ↗First posted 9 Oct 2023cs.CLcited by 30

Not on the map. Why?

Abstract (arXiv)

Despite the impressive performance of Large Language Models (LLM) for various natural language processing tasks, little is known about their comprehension of geographic data and related ability to facilitate informed geospatial decision-making. This paper investigates the extent of geospatial knowledge, awareness, and reasoning abilities encoded within such pretrained LLMs. With a focus on autoregressive language models, we devise experimental approaches related to (i) probing LLMs for geo-coordinates to assess geospatial knowledge, (ii) using geospatial and non-geospatial prepositions to gauge their geospatial awareness, and (iii) utilizing a multidimensional scaling (MDS) experiment to assess the models' geospatial reasoning capabilities and to determine locations of cities based on prompting. Our results confirm that it does not only take larger, but also more sophisticated LLMs to synthesize geospatial knowledge from textual information. As such, this research contributes to understanding the potential and limitations of LLMs in dealing with geospatial information.

Key facts. Cited by 30 papers in the map (data through 7 Oct 2026). Builds on 15 of its 30 references that are in the map. First posted on arXiv 9 Oct 2023.

Builds on

15 of 30 references.

References · 15

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2020Scaling Laws for Neural Language Modelscited by 6,411
  5. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,346
  6. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  7. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,201
  8. 2019Language Models as Knowledge Bases?cited by 1,922
  9. 2020Stanza: A Python Natural Language Processing Toolkit for Many Human Languagescited by 719
  10. 2021P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and Taskscited by 686
  11. 2019CCNet: Extracting High Quality Monolingual Datasets from Web Crawl Datacited by 544
  12. 2018Dissecting Contextual Word Embeddings: Architecture and Representationcited by 146
3 more
  1. 2021Analyzing Commonsense Emergence in Few-shot Knowledge Modelscited by 34
  2. 2021Relational World Knowledge Representation in Contextual Language Models: A Reviewcited by 31
  3. 2021Do Language Models Know the Way to Rome?cited by 18

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 10; 2025: 13; 2026: 6.

30 papers.

Sort:

By area:

  1. 2024CityGPT: Empowering Urban Spatial Cognition of Large Language Modelscited by 29
  2. 2024CityBench: Evaluating the Capabilities of Large Language Models for Urban Taskscited by 26
  3. 2024Towards Urban General Intelligence: A Review and Outlook of Urban Foundation Modelscited by 23
  4. 2024UrbanKGent: A Unified Large Language Model Agent Framework for Urban Knowledge Graph Constructioncited by 15
  5. 2025AI Sees Your Location, But With A Bias Toward The Wealthy Worldcited by 14
  6. 2023GeoLocator: a location-integrated large multimodal model for inferring geo-privacycited by 13
  7. 2024Urban Mobility Assessment Using LLMscited by 13
  8. 2025AeroDuo: Aerial Duo for UAV-based Vision and Language Navigationcited by 13
  9. 2025GeoBenchX: Benchmarking LLMs in Agent Solving Multistep Geospatial Taskscited by 11
  10. 2025A Survey of Large Language Model-Powered Spatial Intelligence Across Scales: Advances in Embodied Agents, Smart Cities, and Earth Sciencecited by 11
  11. 2024Richer Output for Richer Countries: Uncovering Geographical Disparities in Generated Stories and Travel Recommendationscited by 9
  12. 2024Casper: Prompt Sanitization for Protecting User Privacy in Web-Based Large Language Modelscited by 8
18 more
  1. 2024Evaluation of Code LLMs on Geospatial Code Generationcited by 8
  2. 2024Quantifying Geospatial in the Common Crawl Corpuscited by 4
  3. 2025Urban Computing in the Era of Large Language Modelscited by 2
  4. 2025Geospatial Mechanistic Interpretability of Large Language Modelscited by 2
  5. 2025The World As Large Language Models See It: Exploring the reliability of LLMs in representing geographical featurescited by 1
  6. 2025GeoResponder: Towards Building Geospatial LLMs for Time-Critical Disaster Responsecited by 1
  7. 2025Georeferencing complex relative locality descriptions with large language modelscited by 1
  8. 2026Benchmarks for Vision-Language Models in Urban Perception Should Be Reliability-Aware and Negotiatedcited by 1
  9. 2024MALAMUTE: A Multilingual, Highly-granular, Template-free, Education-based Probing Datasetcited by 0
  10. 2025GeoGrid-Bench: Can Foundation Models Understand Multimodal Gridded Geo-Spatial Data?cited by 0
  11. 2025TurnBack: A Geospatial Route Cognition Benchmark for Large Language Models through Reverse Routecited by 0
  12. 2025Coordinates from Context: Using LLMs to Ground Complex Location Referencescited by 0
  13. 2025CartoMapQA: A Fundamental Benchmark Dataset Evaluating Vision-Language Models on Cartographic Map Understandingcited by 0
  14. 2026GTPred: Benchmarking MLLMs for Interpretable Geo-localization and Time-of-capture Predictioncited by 0
  15. 2026Much of Geospatial Web Search Is Beyond Traditional GIScited by 0
  16. 2026Assessing the Geographic Diversity of AI's Platial Representations in Image Generationcited by 0
  17. 2026Graph-Enhanced Large Language Models for Spatial Searchcited by 0
  18. 2026Is Your Neighborhood Safe? Place-based Stigma in Large Language Models' Urban Safety Judgmentscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Can Large Language Models Integrate Spatial Data? Empirical Insights into Reasoning Strengths and Computational Weaknesses
  2. 2023GPT4GEO: How a Language Model Sees the World's Geography
  3. 2025Around the World in 24 Hours: Probing LLM Knowledge of Time and Place
  4. 2026GPSBench: Do Large Language Models Understand GPS Coordinates?
  5. 2026MapVerse: A Benchmark for Geospatial Question Answering on Diverse Real-World Maps
  6. 2024GeoReasoner: Reasoning On Geospatially Grounded Context For Natural Language Understanding
  7. 2025Comparative Evaluation of Prompting and Fine-Tuning for Applying Large Language Models to Grid-Structured Geospatial Data
  8. 2026MultiGlobeQA: A Multilingual and Globally Diverse Benchmark for Geospatial Reasoning