mlsanityData to 14 Oct 2026

Large Multi-modal Model Cartographic Map Comprehension for Textual Locality Georeferencing

arXiv 2507.08575 ↗First posted 11 Jul 2025cs.AI cs.CL cs.CVcited by 0

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

Abstract (arXiv)

Millions of biological sample records collected in the last few centuries archived in natural history collections are un-georeferenced. Georeferencing complex locality descriptions associated with these collection samples is a highly labour-intensive task collection agencies struggle with. None of the existing automated methods exploit maps that are an essential tool for georeferencing complex relations. We present preliminary experiments and results of a novel method that exploits multi-modal capabilities of recent Large Multi-Modal Models (LMM). This method enables the model to visually contextualize spatial relations it reads in the locality description. We use a grid-based approach to adapt these auto-regressive models for this task in a zero-shot setting. Our experiments conducted on a small manually annotated dataset show impressive results for our approach (~1 km Average distance error) compared to uni-modal georeferencing with Large Language Models and existing georeferencing tools. The paper also discusses the findings of the experiments in light of an LMM's ability to comprehend fine-grained maps. Motivated by these results, a practical framework is proposed to integrate this method into a georeferencing workflow.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 31 of its 69 references that are in the map. First posted on arXiv 11 Jul 2025.

Builds on

31 of 69 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023PaLM-E: An Embodied Multimodal Language Model6 Mar

Other references · 30

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  5. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  6. 2023Visual Instruction Tuningcited by 7,869
  7. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  8. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  9. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
  10. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,322
  11. 2015VQA: Visual Question Answeringcited by 2,506
  12. 2025Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scalingcited by 812
18 more
  1. 2022Automatic Chain of Thought Prompting in Large Language Modelscited by 688
  2. 2023A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPTcited by 619
  3. 2018A Survey on Deep Learning for Named Entity Recognitioncited by 282
  4. 2024MMT-Bench: A Comprehensive Multimodal Benchmark for Evaluating Large Vision-Language Models Towards Multitask AGIcited by 193
  5. 2023Improving Multimodal Datasets with Image Captioningcited by 93
  6. 2024TableVQA-Bench: A Visual Question Answering Benchmark on Multiple Table Domainscited by 69
  7. 2024Towards Long-Horizon Vision-Language Navigation: Platform, Benchmark and Methodcited by 67
  8. 2023GPT-RE: In-context Learning for Relation Extraction using Large Language Modelscited by 62
  9. 2023Enhancing Zero-Shot Chain-of-Thought Reasoning in Large Language Models through Logiccited by 35
  10. 2018Anaphora and Coreference Resolution: A Reviewcited by 31
  11. 2024M5 -- A Diverse Benchmark to Assess the Performance of Large Multimodal Models Across Multilingual and Multicultural Vision-Language Taskscited by 27
  12. 2022Global Pointer: Novel Efficient Span-based Approach for Named Entity Recognitioncited by 15
  13. 2022A Transformer-based Framework for POI-level Social Post Geolocationcited by 13
  14. 2024Correctness Comparison of ChatGPT-4, Gemini, Claude-3, and Copilot for Spatial Taskscited by 10
  15. 2020Enhancing spatial and textual analysis with EUPEG: an extensible and unified platform for evaluating geoparserscited by 9
  16. 2024Pixels and Predictions: Potential of GPT-4V in Meteorological Imagery Analysis and Forecast Communicationcited by 4
  17. 2024In-Context Learning for Few-Shot Nested Named Entity Recognitioncited by 3
  18. 2016Deep Embedding for Spatial Role Labelingcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Georeferencing complex relative locality descriptions with large language models
  2. 2025Geospatial Mechanistic Interpretability of Large Language Models
  3. 2026LMM Modality Transfer: A Pre-requisite for Autonomous GIS Agents
  4. 2026What Words Keep of a Place: Zero-Shot Language Reasoning for Cross-View Geo-Localization
  5. 2026MapVerse: A Benchmark for Geospatial Question Answering on Diverse Real-World Maps
  6. 2025CartoMapQA: A Fundamental Benchmark Dataset Evaluating Vision-Language Models on Cartographic Map Understanding
  7. 2026Exploring Geographic Relative Space in Large Language Models through Activation Patching
  8. 2025Linear Spatial World Models Emerge in Large Language Models