mlsanityData to 14 Oct 2026

GPT4GEO: How a Language Model Sees the World's Geography

arXiv 2306.00020 ↗First posted 30 May 2023cs.CL cs.AI cs.LGcited by 68

On the map: World models → World models for language agents

Abstract (arXiv)

Large language models (LLMs) have shown remarkable capabilities across a broad range of tasks involving question answering and the generation of coherent text and code. Comprehensively understanding the strengths and weaknesses of LLMs is beneficial for safety, downstream applications and improving performance. In this work, we investigate the degree to which GPT-4 has acquired factual geographic knowledge and is capable of using this knowledge for interpretative reasoning, which is especially important for applications that involve geographic data, such as geospatial analysis, supply chain management, and disaster response. To this end, we design and conduct a series of diverse experiments, starting from factual tasks such as location, distance and elevation estimation to more complex questions such as generating country outlines and travel networks, route finding under constraints and supply chain analysis. We provide a broad characterisation of what GPT-4 (without plugins or Internet access) knows about the world, highlighting both potentially surprising capabilities but also limitations.

Key facts. Cited by 68 papers in the map (data through 14 Oct 2026). Placed in World models for language agents, one of 11 directions in World models. Builds on 27 of its 63 references that are in the map. First posted on arXiv 30 May 2023.

Builds on

27 of 63 references.

Closest ancestors · 1

Same direction, newest first

  1. 2020Language Models are Few-Shot Learners28 May

Other references · 26

  1. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  2. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  3. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,592
  4. 2022Survey of Hallucination in Natural Language Generationcited by 2,075
  5. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,984
  6. 2016Concrete Problems in AI Safetycited by 1,642
  7. 2023Capabilities of GPT-4 on Medical Challenge Problemscited by 638
  8. 2023Is ChatGPT Good at Search? Investigating Large Language Models as Re-Ranking Agentscited by 360
  9. 2023Evaluating the Logical Reasoning Ability of ChatGPT and GPT-4cited by 202
  10. 2023Model evaluation for extreme riskscited by 177
  11. 2023How well do Large Language Models perform in Arithmetic tasks?cited by 130
  12. 2023Speak, Memory: An Archaeology of Books Known to ChatGPT/GPT-4cited by 125
14 more
  1. 2022Extracting Cultural Commonsense Knowledge at Scalecited by 76
  2. 2023Can GPT-4 Perform Neural Architecture Search?cited by 74
  3. 2023Evaluating GPT-4 and ChatGPT on Japanese Medical Licensing Examinationscited by 63
  4. 2023Autonomous GIS: the next-generation AI-powered GIScited by 53
  5. 2023Spear Phishing With Large Language Modelscited by 52
  6. 2023AI-assisted coding: Experiments with GPT-4cited by 43
  7. 2023Are Large Language Models Ready for Healthcare? A Comparative Study on Clinical Language Understandingcited by 40
  8. 2023chatClimate: Grounding Conversational AI in Climate Sciencecited by 32
  9. 2022GeoMLAMA: Geo-Diverse Commonsense Probing on Multilingual Pre-Trained Language Modelscited by 30
  10. 2023Evaluation of GPT-3.5 and GPT-4 for supporting real-world information needs in healthcare deliverycited by 30
  11. 2022ERNIE-GeoL: A Geography-and-Language Pre-trained Model and its Applications in Baidu Mapscited by 25
  12. 2023SATIN: A Multi-Task Metadataset for Classifying Satellite Imagery using Vision-Language Modelscited by 15
  13. 2023GersteinLab at MEDIQA-Chat 2023: Clinical Note Summarization from Doctor-Patient Conversations through Fine-tuning and In-context Learningcited by 9
  14. 2023The Multimodal And Modular Ai Chef: Complex Recipe Generation From Imagerycited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 8; 2024: 31; 2025: 20; 2026: 9.

68 papers.

Sort:

By area:

  1. 2024A Survey on Knowledge Distillation of Large Language Modelscited by 284
  2. 2024Understanding World or Predicting Future? A Comprehensive Survey of World Modelscited by 166
  3. 2024CultureBank: An Online Community-Driven Knowledge Base Towards Culturally Aware Language Technologiescited by 98
  4. 2024RS-Agent: Automating Remote Sensing Tasks through Intelligent Agentcited by 49
  5. 2024SciFIBench: Benchmarking Large Multimodal Models for Scientific Figure Interpretationcited by 48
  6. 2023GeoGPT: Understanding and Processing Geospatial Tasks through An Autonomous GPTcited by 41
  7. 2024Towards Vision-Language Geo-Foundation Model: A Surveycited by 36
  8. 2023Charting New Territories: Exploring the Geographic and Geospatial Capabilities of Multimodal LLMscited by 34
  9. 2024Deep Learning for Cross-Domain Data Fusion in Urban Computing: Taxonomy, Advances, and Outlookcited by 33
  10. 2024CityGPT: Empowering Urban Spatial Cognition of Large Language Modelscited by 30
  11. 2024Towards Urban General Intelligence: A Review and Outlook of Urban Foundation Modelscited by 23
  12. 2024CHOICE: Benchmarking the Remote Sensing Capabilities of Large Vision-Language Modelscited by 22
38 more
  1. 2023WorldSense: A Synthetic Benchmark for Grounded Reasoning in Large Language Modelscited by 20
  2. 2024Where to Move Next: Zero-shot Generalization of LLMs for Next POI Recommendationcited by 17
  3. 2024TRIP-PAL: Travel Planning with Guarantees by Combining Large Language Models and Automated Plannerscited by 17
  4. 2024GAMEBoT: Transparent Assessment of LLM Reasoning in Gamescited by 17
  5. 2023On the Promises and Challenges of Multimodal Foundation Models for Geographical, Environmental, Agricultural, and Urban Planning Applicationscited by 15
  6. 2025AI Sees Your Location, But With A Bias Toward The Wealthy Worldcited by 14
  7. 2025GeoBenchX: Benchmarking LLMs in Agent Solving Multistep Geospatial Taskscited by 14
  8. 2023When Geoscience Meets Foundation Models: Towards General Geoscience Artificial Intelligence Systemcited by 13
  9. 2023GeoLocator: a location-integrated large multimodal model for inferring geo-privacycited by 13
  10. 2025Spatial-RAG: Spatial Retrieval Augmented Generation for Real-World Geospatial Reasoning Questionscited by 13
  11. 2024Combining Observational Data and Language for Species Range Estimationcited by 12
  12. 2024Needle Threading: Can LLMs Follow Threads through Near-Million-Scale Haystacks?cited by 12
  13. 2024STBench: Assessing the Ability of Large Language Models in Spatio-Temporal Analysiscited by 11
  14. 2024Towards Interactive Global Geolocation Assistantcited by 11
  15. 2024Self-Supervised Representation Learning for Geospatial Objects: A Surveycited by 9
  16. 2024ShapefileGPT: A Multi-Agent Large Language Model Framework for Automated Shapefile Processingcited by 9
  17. 2024Evaluation of Code LLMs on Geospatial Code Generationcited by 8
  18. 2024Evaluation of Geographical Distortions in Language Modelscited by 6
  19. 2024Distortions in Judged Spatial Relations in Large Language Modelscited by 5
  20. 2024From Neurons to Neutrons: A Case Study in Interpretabilitycited by 5
  21. 2025Where on Earth? A Vision-Language Benchmark for Probing Model Geolocation Skills Across Scalescited by 5
  22. 2026GeoAgentBench: A Dynamic Execution Benchmark for Tool-Augmented Agents in Spatial Analysiscited by 5
  23. 2023Core Building Blocks: Next Gen Geo Spatial GPT Applicationcited by 4
  24. 2024V-RoAst: Visual Road Assessment. Can VLM be a Road Safety Assessor Using the iRAP Standard?cited by 4
  25. 2025CAMS: A CityGPT-Powered Agentic Framework for Urban Human Mobility Simulationcited by 4
  26. 2024Surveying Attitudinal Alignment Between Large Language Models Vs. Humans Towards 17 Sustainable Development Goalscited by 3
  27. 2025MapStory: Prototyping Editable Map Animations with LLM Agentscited by 3
  28. 2025Mitigating Geospatial Knowledge Hallucination in Large Language Models: Benchmarking and Dynamic Factuality Aligningcited by 3
  29. 2025Towards Interpretable Geo-localization: a Concept-Aware Global Image-GPS Alignment Frameworkcited by 3
  30. 2025Urban Computing in the Era of Large Language Modelscited by 2
  31. 2025Geospatial Mechanistic Interpretability of Large Language Modelscited by 2
  32. 2025Geography-Aware Large Language Models for Next POI Recommendationcited by 2
  33. 2023Generative AI May Prefer to Present National-level Characteristics of Cities Based on Stereotypical Geographic Impressions at the Continental Levelcited by 1
  34. 2024Knowledge-based Consistency Testing of Large Language Modelscited by 1
  35. 2024Transit Pulse: Utilizing Social Media as a Source for Customer Feedback and Information Extraction with Large Language Modelcited by 1
  36. 2024Advancing Large Language Models for Spatiotemporal and Semantic Association Mining of Similar Environmental Eventscited by 1
  37. 2025AutoCas: Autoregressive Cascade Predictor in Social Networks via Large Language Modelscited by 1
  38. 2025The World As Large Language Models See It: Exploring the reliability of LLMs in representing geographical featurescited by 1

50 most cited of 68 All 68 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023Are Large Language Models Geospatially Knowledgeable?
  2. 2026MapVerse: A Benchmark for Geospatial Question Answering on Diverse Real-World Maps
  3. 2026GPSBench: Do Large Language Models Understand GPS Coordinates?
  4. 2025Around the World in 24 Hours: Probing LLM Knowledge of Time and Place
  5. 2025Foundation Models for Geospatial Reasoning: Assessing Capabilities of Large Language Models in Understanding Geometries and Topological Spatial Relations
  6. 2026Exploring Geographic Relative Space in Large Language Models through Activation Patching
  7. 2025Understanding the Geospatial Reasoning Capabilities of LLMs: A Trajectory Recovery Perspective
  8. 2025Can Large Language Models Integrate Spatial Data? Empirical Insights into Reasoning Strengths and Computational Weaknesses