# GPT4GEO: How a Language Model Sees the World's Geography

arXiv: https://arxiv.org/abs/2306.00020 · HTML: https://mlsanity.com/paper/2306.00020/

## Key facts

- Cited by 68 papers in the map (data through 14 Oct 2026).
- Placed in World models for language agents, one of 11 directions in World models.
- Builds on 27 of its 63 references that are in the map.
- First posted on arXiv 30 May 2023.

## Placed in

- World models / [World models for language agents](https://mlsanity.com/wm/world-models-for-language-agents/)

## Builds on (most cited first, 27 of 27)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [Sparks of Artificial General Intelligence: Early experiments with GPT-4](https://mlsanity.com/paper/2303.12712/) (2023)
- [Survey of Hallucination in Natural Language Generation](https://mlsanity.com/paper/2202.03629/) (2022)
- [BLOOM: A 176B-Parameter Open-Access Multilingual Language Model](https://mlsanity.com/paper/2211.05100/) (2022)
- [Concrete Problems in AI Safety](https://mlsanity.com/paper/1606.06565/) (2016)
- [Capabilities of GPT-4 on Medical Challenge Problems](https://mlsanity.com/paper/2303.13375/) (2023)
- [Is ChatGPT Good at Search? Investigating Large Language Models as Re-Ranking Agents](https://mlsanity.com/paper/2304.09542/) (2023)
- [Evaluating the Logical Reasoning Ability of ChatGPT and GPT-4](https://mlsanity.com/paper/2304.03439/) (2023)
- [Model evaluation for extreme risks](https://mlsanity.com/paper/2305.15324/) (2023)
- [How well do Large Language Models perform in Arithmetic tasks?](https://mlsanity.com/paper/2304.02015/) (2023)
- [Speak, Memory: An Archaeology of Books Known to ChatGPT/GPT-4](https://mlsanity.com/paper/2305.00118/) (2023)
- [Extracting Cultural Commonsense Knowledge at Scale](https://mlsanity.com/paper/2210.07763/) (2022)
- [Can GPT-4 Perform Neural Architecture Search?](https://mlsanity.com/paper/2304.10970/) (2023)
- [Evaluating GPT-4 and ChatGPT on Japanese Medical Licensing Examinations](https://mlsanity.com/paper/2303.18027/) (2023)
- [Autonomous GIS: the next-generation AI-powered GIS](https://mlsanity.com/paper/2305.06453/) (2023)
- [Spear Phishing With Large Language Models](https://mlsanity.com/paper/2305.06972/) (2023)
- [AI-assisted coding: Experiments with GPT-4](https://mlsanity.com/paper/2304.13187/) (2023)
- [Are Large Language Models Ready for Healthcare? A Comparative Study on Clinical Language Understanding](https://mlsanity.com/paper/2304.05368/) (2023)
- [chatClimate: Grounding Conversational AI in Climate Science](https://mlsanity.com/paper/2304.05510/) (2023)
- [GeoMLAMA: Geo-Diverse Commonsense Probing on Multilingual Pre-Trained Language Models](https://mlsanity.com/paper/2205.12247/) (2022)
- [Evaluation of GPT-3.5 and GPT-4 for supporting real-world information needs in healthcare delivery](https://mlsanity.com/paper/2304.13714/) (2023)
- [ERNIE-GeoL: A Geography-and-Language Pre-trained Model and its Applications in Baidu Maps](https://mlsanity.com/paper/2203.09127/) (2022)
- [SATIN: A Multi-Task Metadataset for Classifying Satellite Imagery using Vision-Language Models](https://mlsanity.com/paper/2304.11619/) (2023)
- [GersteinLab at MEDIQA-Chat 2023: Clinical Note Summarization from Doctor-Patient Conversations through Fine-tuning and In-context Learning](https://mlsanity.com/paper/2305.05001/) (2023)
- [The Multimodal And Modular Ai Chef: Complex Recipe Generation From Imagery](https://mlsanity.com/paper/2304.02016/) (2023)

## Built on by (most cited first, 30 of 68)

- [A Survey on Knowledge Distillation of Large Language Models](https://mlsanity.com/paper/2402.13116/) (2024)
- [Understanding World or Predicting Future? A Comprehensive Survey of World Models](https://mlsanity.com/paper/2411.14499/) (2024)
- [CultureBank: An Online Community-Driven Knowledge Base Towards Culturally Aware Language Technologies](https://mlsanity.com/paper/2404.15238/) (2024)
- [RS-Agent: Automating Remote Sensing Tasks through Intelligent Agent](https://mlsanity.com/paper/2406.07089/) (2024)
- [SciFIBench: Benchmarking Large Multimodal Models for Scientific Figure Interpretation](https://mlsanity.com/paper/2405.08807/) (2024)
- [GeoGPT: Understanding and Processing Geospatial Tasks through An Autonomous GPT](https://mlsanity.com/paper/2307.07930/) (2023)
- [Towards Vision-Language Geo-Foundation Model: A Survey](https://mlsanity.com/paper/2406.09385/) (2024)
- [Charting New Territories: Exploring the Geographic and Geospatial Capabilities of Multimodal LLMs](https://mlsanity.com/paper/2311.14656/) (2023)
- [Deep Learning for Cross-Domain Data Fusion in Urban Computing: Taxonomy, Advances, and Outlook](https://mlsanity.com/paper/2402.19348/) (2024)
- [CityGPT: Empowering Urban Spatial Cognition of Large Language Models](https://mlsanity.com/paper/2406.13948/) (2024)
- [Towards Urban General Intelligence: A Review and Outlook of Urban Foundation Models](https://mlsanity.com/paper/2402.01749/) (2024)
- [CHOICE: Benchmarking the Remote Sensing Capabilities of Large Vision-Language Models](https://mlsanity.com/paper/2411.18145/) (2024)
- [WorldSense: A Synthetic Benchmark for Grounded Reasoning in Large Language Models](https://mlsanity.com/paper/2311.15930/) (2023)
- [Where to Move Next: Zero-shot Generalization of LLMs for Next POI Recommendation](https://mlsanity.com/paper/2404.01855/) (2024)
- [TRIP-PAL: Travel Planning with Guarantees by Combining Large Language Models and Automated Planners](https://mlsanity.com/paper/2406.10196/) (2024)
- [GAMEBoT: Transparent Assessment of LLM Reasoning in Games](https://mlsanity.com/paper/2412.13602/) (2024)
- [On the Promises and Challenges of Multimodal Foundation Models for Geographical, Environmental, Agricultural, and Urban Planning Applications](https://mlsanity.com/paper/2312.17016/) (2023)
- [AI Sees Your Location, But With A Bias Toward The Wealthy World](https://mlsanity.com/paper/2502.11163/) (2025)
- [GeoBenchX: Benchmarking LLMs in Agent Solving Multistep Geospatial Tasks](https://mlsanity.com/paper/2503.18129/) (2025)
- [When Geoscience Meets Foundation Models: Towards General Geoscience Artificial Intelligence System](https://mlsanity.com/paper/2309.06799/) (2023)
- [GeoLocator: a location-integrated large multimodal model for inferring geo-privacy](https://mlsanity.com/paper/2311.13018/) (2023)
- [Spatial-RAG: Spatial Retrieval Augmented Generation for Real-World Geospatial Reasoning Questions](https://mlsanity.com/paper/2502.18470/) (2025)
- [Combining Observational Data and Language for Species Range Estimation](https://mlsanity.com/paper/2410.10931/) (2024)
- [Needle Threading: Can LLMs Follow Threads through Near-Million-Scale Haystacks?](https://mlsanity.com/paper/2411.05000/) (2024)
- [STBench: Assessing the Ability of Large Language Models in Spatio-Temporal Analysis](https://mlsanity.com/paper/2406.19065/) (2024)
- [Towards Interactive Global Geolocation Assistant](https://mlsanity.com/paper/2412.08907/) (2024)
- [Self-Supervised Representation Learning for Geospatial Objects: A Survey](https://mlsanity.com/paper/2408.12133/) (2024)
- [ShapefileGPT: A Multi-Agent Large Language Model Framework for Automated Shapefile Processing](https://mlsanity.com/paper/2410.12376/) (2024)
- [Evaluation of Code LLMs on Geospatial Code Generation](https://mlsanity.com/paper/2410.04617/) (2024)
- [Evaluation of Geographical Distortions in Language Models](https://mlsanity.com/paper/2404.17401/) (2024)

## Abstract (arXiv)

Large language models (LLMs) have shown remarkable capabilities across a broad range of tasks involving question answering and the generation of coherent text and code. Comprehensively understanding the strengths and weaknesses of LLMs is beneficial for safety, downstream applications and improving performance. In this work, we investigate the degree to which GPT-4 has acquired factual geographic knowledge and is capable of using this knowledge for interpretative reasoning, which is especially important for applications that involve geographic data, such as geospatial analysis, supply chain management, and disaster response. To this end, we design and conduct a series of diverse experiments, starting from factual tasks such as location, distance and elevation estimation to more complex questions such as generating country outlines and travel networks, route finding under constraints and supply chain analysis. We provide a broad characterisation of what GPT-4 (without plugins or Internet access) knows about the world, highlighting both potentially surprising capabilities but also limitations.