Mapping the City Through the Lens of Language Models
Not on the map. Why?
Abstract (arXiv)
Language models often complete an underspecified reference to a city with unstated assumptions about urban size, form, infrastructure, environment, and function. We measure those assumptions without naming places. Ten open-weight checkpoints rate anonymized profiles derived from real morphological urban centres across 40 audited indicators and seven domains. The design combines constrained probability-based ratings, prespecified reliability screens, lineage-aware aggregation, multiple population weightings, an independent replication sample, and whole-profile validation. The clearest shared tendency favours urban profiles with larger developed area, faster recent growth, greater mapped infrastructure and non-residential capacity, and less sparse form. Most eligible directions recur in the replication data, and direct ratings of complete profiles show moderate agreement with the indicator-wise construction. Geographic differences shrink after accounting for city scale and development, while reliably measured paired tasks indicate that typicality and desirability are often closely aligned. The framework makes an otherwise vague notion of what models regard as an ordinary city empirically traceable. The resulting evidence delineates a shared yet model-dependent portrait of the city through the lens of language models.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 20 of its 38 references that are in the map. First posted on arXiv 4 Aug 2026.
Builds on
20 of 38 references.
References · 20
- 2024The Llama 3 Herd of Modelscited by 14,715
- 2025Qwen3 Technical Reportcited by 8,820
- 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,998
- 2018Datasheets for Datasetscited by 1,386
- 2023Large Language Models are not Fair Evaluatorscited by 1,085
- 2018Model Cards for Model Reportingcited by 1,080
- 2020Beyond Accuracy: Behavioral Testing of NLP models with CheckListcited by 1,006
- 2023Whose Opinions Do Language Models Reflect?cited by 773
- 2020Improving Reproducibility in Machine Learning Research (A Report from the NeurIPS 2019 Reproducibility Program)cited by 322
- 2023Large Language Models Sensitivity to The Order of Options in Multiple-Choice Questionscited by 263
- 2024Large Language Models are Geographically Biasedcited by 93
- 2023This Land is {Your, My} Land: Evaluating Geopolitical Biases in Language Modelscited by 32
8 more
- 2022Geographic and Geopolitical Biases of Language Modelscited by 29
- 2024CityBench: Evaluating the Capabilities of Large Language Models for Urban Taskscited by 27
- 2025GlobalBuildingAtlas: An Open Global and Complete Dataset of Building Polygons, Heights and LoD1 3D Modelscited by 16
- 2024Contextualized Evaluations: Judging Language Model Responses to Underspecified Queriescited by 12
- 2024Pre-Trained Language Models Represent Some Geographic Populations Better Than Otherscited by 6
- 2025UrbanPlanBench: A Comprehensive Urban Planning Benchmark for Evaluating Large Language Modelscited by 5
- 2024Big City Bias: Evaluating the Impact of Metropolitan Size on Computational Job Market Abilities of Language Modelscited by 2
- 2026Culturally uneven urban perception in large language modelscited by 1
Built on by
Not cited yet.