# GeoGLUE: A GeoGraphic Language Understanding Evaluation Benchmark

arXiv: https://arxiv.org/abs/2305.06545 · HTML: https://mlsanity.com/paper/2305.06545/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 36 references that are in the map.
- First posted on arXiv 11 May 2023.

## Builds on (most cited first, 27 of 27)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems](https://mlsanity.com/paper/1905.00537/) (2019)
- [Adversarial NLI: A New Benchmark for Natural Language Understanding](https://mlsanity.com/paper/1910.14599/) (2019)
- [XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalization](https://mlsanity.com/paper/2003.11080/) (2020)
- [ERASER: A Benchmark to Evaluate Rationalized NLP Models](https://mlsanity.com/paper/1911.03429/) (2019)
- [ERNIE: Enhanced Language Representation with Informative Entities](https://mlsanity.com/paper/1905.07129/) (2019)
- [TweetEval: Unified Benchmark and Comparative Evaluation for Tweet Classification](https://mlsanity.com/paper/2010.12421/) (2020)
- [LexGLUE: A Benchmark Dataset for Legal Language Understanding in English](https://mlsanity.com/paper/2110.00976/) (2021)
- [CLUE: A Chinese Language Understanding Evaluation Benchmark](https://mlsanity.com/paper/2004.05986/) (2020)
- [Latent Hatred: A Benchmark for Understanding Implicit Hate Speech](https://mlsanity.com/paper/2109.05322/) (2021)
- [CORD-19: The COVID-19 Open Research Dataset](https://mlsanity.com/paper/2004.10706/) (2020)
- [VALSE: A Task-Independent Benchmark for Vision and Language Models Centered on Linguistic Phenomena](https://mlsanity.com/paper/2112.07566/) (2021)
- [StructBERT: Incorporating Language Structures into Pre-training for Deep Language Understanding](https://mlsanity.com/paper/1908.04577/) (2019)
- [IndoNLU: Benchmark and Resources for Evaluating Indonesian Natural Language Understanding](https://mlsanity.com/paper/2009.05387/) (2020)
- [CBLUE: A Chinese Biomedical Language Understanding Evaluation Benchmark](https://mlsanity.com/paper/2106.08087/) (2021)
- [IndoLEM and IndoBERT: A Benchmark Dataset and Pre-trained Language Model for Indonesian NLP](https://mlsanity.com/paper/2011.00677/) (2020)
- [RussianSuperGLUE: A Russian Language Understanding Evaluation Benchmark](https://mlsanity.com/paper/2010.15925/) (2020)
- [KLEJ: Comprehensive Benchmark for Polish Language Understanding](https://mlsanity.com/paper/2005.00630/) (2020)
- [XL-WiC: A Multilingual Benchmark for Evaluating Semantic Contextualization](https://mlsanity.com/paper/2010.06478/) (2020)
- [EQUATE: A Benchmark Evaluation Framework for Quantitative Reasoning in Natural Language Inference](https://mlsanity.com/paper/1901.03735/) (2019)
- [NEZHA: Neural Contextualized Representation for Chinese Language Understanding](https://mlsanity.com/paper/1909.00204/) (2019)
- [KorNLI and KorSTS: New Benchmark Datasets for Korean Natural Language Understanding](https://mlsanity.com/paper/2004.03289/) (2020)
- [SpaBERT: A Pretrained Language Model from Geographic Data for Geo-Entity Representation](https://mlsanity.com/paper/2210.12213/) (2022)
- [GeoSQA: A Benchmark for Scenario-based Question Answering in the Geography Domain at High School Level](https://mlsanity.com/paper/1908.07855/) (2019)
- [QuoteR: A Benchmark of Quote Recommendation for Writing](https://mlsanity.com/paper/2202.13145/) (2022)

## Built on by (most cited first, 5 of 5)

- [CityGPT: Empowering Urban Spatial Cognition of Large Language Models](https://mlsanity.com/paper/2406.13948/) (2024)
- [GeoAnalystBench: A GeoAI benchmark for assessing large language models for spatial analysis workflow and code generation](https://mlsanity.com/paper/2509.05881/) (2025)
- [CourseGPT-zh: an Educational Large Language Model Based on Knowledge Distillation Incorporating Prompt Optimization](https://mlsanity.com/paper/2405.04781/) (2024)
- [GPSBench: Do Large Language Models Understand GPS Coordinates?](https://mlsanity.com/paper/2602.16105/) (2026)
- [AddrLLM: Address Rewriting via Large Language Model on Nationwide Logistics Data](https://mlsanity.com/paper/2411.13584/) (2024)

## Abstract (arXiv)

With a fast developing pace of geographic applications, automatable and intelligent models are essential to be designed to handle the large volume of information. However, few researchers focus on geographic natural language processing, and there has never been a benchmark to build a unified standard. In this work, we propose a GeoGraphic Language Understanding Evaluation benchmark, named GeoGLUE. We collect data from open-released geographic resources and introduce six natural language understanding tasks, including geographic textual similarity on recall, geographic textual similarity on rerank, geographic elements tagging, geographic composition analysis, geographic where what cut, and geographic entity alignment. We also pro vide evaluation experiments and analysis of general baselines, indicating the effectiveness and significance of the GeoGLUE benchmark.