mlsanityData to 14 Oct 2026

Do Language Models Know the Way to Rome?

arXiv 2109.07971 ↗First posted 16 Sep 2021cs.CLcited by 18

Not on the map. Why?

Abstract (arXiv)

The global geometry of language models is important for a range of applications, but language model probes tend to evaluate rather local relations, for which ground truths are easily obtained. In this paper we exploit the fact that in geography, ground truths are available beyond local relations. In a series of experiments, we evaluate the extent to which language model representations of city and country names are isomorphic to real-world geography, e.g., if you tell a language model where Paris and Berlin are, does it know the way to Rome? We find that language models generally encode limited geographic information, but with larger models performing the best, suggesting that geographic knowledge can be induced from higher-order co-occurrence statistics.

Key facts. Cited by 18 papers in the map (data through 14 Oct 2026). Builds on 19 of its 32 references that are in the map. First posted on arXiv 16 Sep 2021.

Builds on

19 of 32 references.

References · 19

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  3. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,375
  4. 2019Language Models as Knowledge Bases?cited by 1,936
  5. 2019BERT Rediscovers the Classical NLP Pipelinecited by 1,059
  6. 2019How Can We Know What Language Models Know?cited by 959
  7. 2019How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddingscited by 784
  8. 2020How Much Knowledge Can You Pack Into the Parameters of a Language Model?cited by 719
  9. 2019Designing and Interpreting Probes with Control Taskscited by 640
  10. 2019Visualizing and Measuring the Geometry of BERTcited by 216
  11. 2019Commonsense Knowledge Mining from Pretrained Modelscited by 189
  12. 2014SimLex-999: Evaluating Semantic Models with (Genuine) Similarity Estimationcited by 172
7 more
  1. 2021Can Language Models Encode Perceptual Structure Without Grounding? A Case Study in Colorcited by 119
  2. 2019Do Neural Language Representations Learn Physical Commonsense?cited by 62
  3. 2018On the Limitations of Unsupervised Bilingual Dictionary Inductioncited by 61
  4. 2019Blackbox meets blackbox: Representational Similarity and Stability Analysis of Neural Language Models and Brainscited by 38
  5. 2020Are All Good Word Vector Spaces Isomorphic?cited by 37
  6. 2017Are distributional representations ready for the real world? Evaluating word vectors for grounded perceptual meaningcited by 19
  7. 2016Intrinsic Subspace Evaluation of Word Embedding Representationscited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 1; 2023: 4; 2024: 2; 2025: 9; 2026: 2.

18 papers.

Sort:

By area:

  1. 2023Language Models Represent Space and Timecited by 316
  2. 2023Eight Things to Know about Large Language Modelscited by 73
  3. 2023Give Me the Facts! A Survey on Factual Knowledge Probing in Pre-trained Language Modelscited by 32
  4. 2023Are Large Language Models Geospatially Knowledgeable?cited by 30
  5. 2024What Do Language Models Hear? Probing for Auditory Representations in Language Modelscited by 15
  6. 2024Monotonic Representation of Numeric Properties in Language Modelscited by 15
  7. 2025Spatial-RAG: Spatial Retrieval Augmented Generation for Real-World Geospatial Reasoning Questionscited by 13
  8. 2025Around the World in 24 Hours: Probing LLM Knowledge of Time and Placecited by 7
  9. 2025Linear Spatial World Models Emerge in Large Language Modelscited by 6
  10. 2025On Entity Identification in Language Modelscited by 3
  11. 2025Geospatial Mechanistic Interpretability of Large Language Modelscited by 2
  12. 2026Where meaning lives: Layer-wise accessibility of psycholinguistic features in encoder and decoder language modelscited by 2
6 more
  1. 2022Ered: Enhanced Text Representations with Entities and Descriptionscited by 1
  2. 2025Can LLMs Learn to Map the World from Local Descriptions?cited by 1
  3. 2025Towards Applying Large Language Models to Complement Single-Cell Foundation Modelscited by 1
  4. 2025Explaining Black-box Language Models with Knowledge Probing Systems: A Post-hoc Explanation Perspectivecited by 1
  5. 2025Interpreting Multi-Attribute Confounding through Numerical Attributes in Large Language Modelscited by 0
  6. 2026Recovering Temporal and Geographic Signals from Language Model Embeddingscited by 0