mlsanityData to 14 Oct 2026

GeoGrid-Bench: Can Foundation Models Understand Multimodal Gridded Geo-Spatial Data?

arXiv 2505.10714 ↗First posted 15 May 2025cs.CLcited by 1

Not on the map. Why?

Abstract (arXiv)

We present GeoGrid-Bench, a benchmark designed to evaluate the ability of foundation models to understand geo-spatial data in the grid structure. Geo-spatial datasets pose distinct challenges due to their dense numerical values, strong spatial and temporal dependencies, and unique multimodal representations including tabular data, heatmaps, and geographic visualizations. To assess how foundation models can support scientific research in this domain, GeoGrid-Bench features large-scale, real-world data covering 16 climate variables across 150 locations and extended time frames. The benchmark includes approximately 3,200 question-answer pairs, systematically generated from 8 domain expert-curated templates to reflect practical tasks encountered by human scientists. These range from basic queries at a single location and time to complex spatiotemporal comparisons across regions and periods. Our evaluation reveals that vision-language models perform best overall, and we provide a fine-grained analysis of the strengths and limitations of different foundation models in different geo-spatial tasks. This benchmark offers clearer insights into how foundation models can be effectively applied to geo-spatial data analysis and used to support scientific research.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 53 of its 76 references that are in the map. First posted on arXiv 15 May 2025.

Builds on

53 of 76 references.

References · 53

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  3. 2025Qwen2.5-VL Technical Reportcited by 5,587
  4. 2024GPT-4o System Cardcited by 4,588
  5. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
  6. 2024A Survey on Large Language Models for Code Generationcited by 658
  7. 2021FinQA: A Dataset of Numerical Reasoning over Financial Datacited by 562
  8. 2015Compositional Semantic Parsing on Semi-Structured Tablescited by 515
  9. 2019TabFact: A Large-scale Dataset for Table-based Fact Verificationcited by 428
  10. 2023ClimaX: A foundation model for weather and climatecited by 337
  11. 2023GeoChat: Grounded Large Vision-Language Model for Remote Sensingcited by 328
  12. 2020ToTTo: A Controlled Table-To-Text Generation Datasetcited by 268
38 more
  1. 2020HybridQA: A Dataset of Multi-Hop Question Answering over Tabular and Textual Datacited by 264
  2. 2022Binding Language Models in Symbolic Languagescited by 228
  3. 2024AI Agents That Mattercited by 219
  4. 2024Chain-of-Table: Evolving Tables in the Reasoning Chain for Table Understandingcited by 188
  5. 2021FEVEROUS: Fact Extraction and VERification Over Unstructured and Structured informationcited by 167
  6. 2024Large Language Models(LLMs) on Tabular Data: Prediction, Generation, and Understanding -- A Surveycited by 159
  7. 2024LHRS-Bot: Empowering Remote Sensing with VGI-Enhanced Large Multimodal Language Modelcited by 138
  8. 2023TableLlama: Towards Open Large Generalist Models for Tablescited by 137
  9. 2023GEO-Bench: Toward Foundation Models for Earth Monitoringcited by 124
  10. 2023Table Meets LLM: Can Large Language Models Understand Structured Table Data? A Benchmark and Empirical Studycited by 116
  11. 2025Agentic Large Language Models, a surveycited by 108
  12. 2023On the Opportunities and Challenges of Foundation Models for Geospatial Artificial Intelligencecited by 88
  13. 2023ReAcTable: Enhancing ReAct for Table Question Answeringcited by 76
  14. 2024A Peek into Token Bias: Large Language Models Are Not Yet Genuine Reasonerscited by 73
  15. 2023Table-GPT: Table-tuned GPT for Diverse Table Taskscited by 69
  16. 2023GeoLLM: Extracting Geospatial Knowledge from Large Language Modelscited by 64
  17. 2024Retrieval & Fine-Tuning for In-Context Tabular Modelscited by 59
  18. 2023Autonomous GIS: the next-generation AI-powered GIScited by 53
  19. 2023EarthVQA: Towards Queryable Earth via Relational Reasoning-Based Remote Sensing Visual Question Answeringcited by 52
  20. 2024GEOBench-VLM: Benchmarking Vision-Language Models for Geospatial Taskscited by 51
  21. 2024ClimateGPT: Towards AI Synthesizing Interdisciplinary Research on Climate Changecited by 47
  22. 2023GeoGPT: Understanding and Processing Geospatial Tasks through An Autonomous GPTcited by 41
  23. 2024Tables as Texts or Images: Evaluating the Table Reasoning Ability of LLMs and MLLMscited by 40
  24. 2024Good at captioning, bad at counting: Benchmarking GPT-4V on Earth observation datacited by 39
  25. 2023Text2Analysis: A Benchmark of Table Question Answering with Advanced Data Analysis and Unclear Queriescited by 38
  26. 2023DisasterResponseGPT: Large Language Models for Accelerated Plan of Action Development in Disaster Response Scenarioscited by 34
  27. 2023chatClimate: Grounding Conversational AI in Climate Sciencecited by 32
  28. 2023Are Large Language Models Geospatially Knowledgeable?cited by 30
  29. 2023UniTabE: A Universal Pretraining Protocol for Tabular Foundation Model in Data Sciencecited by 29
  30. 2024A Survey of Table Reasoning with Large Language Modelscited by 29
  31. 2024An Autonomous GIS Agent Framework for Geospatial Data Retrievalcited by 26
  32. 2023Assessing Large Language Models on Climate Informationcited by 23
  33. 2024Eureka: Evaluating and Understanding Large Foundation Modelscited by 18
  34. 2024Towards Rationality in Language and Multimodal Agents: A Surveycited by 16
  35. 2024WildfireGPT: Tailored Large Language Model for Wildfire Analysiscited by 15
  36. 2024Multi-Agent VQA: Exploring Multi-Agent Foundation Models in Zero-Shot Visual Question Answeringcited by 15
  37. 2025GAIR: Location-Aware Self-Supervised Contrastive Pre-Training with Geo-Aligned Implicit Representationscited by 10
  38. 2024Large Language Models in Fire Engineering: An Examination of Technical Questions Against Domain Knowledgecited by 5

50 most cited of 53 All 53 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Google Earth AI Geospatial Reasoning Agent: Capabilities and a Systematic Evaluationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Scaling Laws for Geospatial Foundation Models: A case study on PhilEO Bench
  2. 2026Earth Science Foundation Models: From Perception to Reasoning and Discovery
  3. 2023When Geoscience Meets Foundation Models: Towards General Geoscience Artificial Intelligence System
  4. 2025GeoAnalystBench: A GeoAI benchmark for assessing large language models for spatial analysis workflow and code generation
  5. 2024Towards Vision-Language Geo-Foundation Model: A Survey
  6. 2026GeoCore-9B: Towards Geo-Aware Generative Foundation Models in Earth Observation
  7. 2026GeoR-Bench: Evaluating Geoscience Visual Reasoning
  8. 2025Towards Scalable and Generalizable Earth Observation Data Mining via Foundation Model Composition