# Geolocation Representation from Large Language Models are Generic Enhancers for Spatio-Temporal Learning

arXiv: https://arxiv.org/abs/2408.12116 · HTML: https://mlsanity.com/paper/2408.12116/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 39 of its 72 references that are in the map.
- First posted on arXiv 22 Aug 2024.

## Builds on (most cited first, 30 of 39)

- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting](https://mlsanity.com/paper/2012.07436/) (2020)
- [A Time Series is Worth 64 Words: Long-term Forecasting with Transformers](https://mlsanity.com/paper/2211.14730/) (2022)
- [Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting](https://mlsanity.com/paper/2106.13008/) (2021)
- [Spatio-Temporal Graph Convolutional Networks: A Deep Learning Framework for Traffic Forecasting](https://mlsanity.com/paper/1709.04875/) (2017)
- [Graph WaveNet for Deep Spatial-Temporal Graph Modeling](https://mlsanity.com/paper/1906.00121/) (2019)
- [One Fits All:Power General Time Series Analysis by Pretrained LM](https://mlsanity.com/paper/2302.11939/) (2023)
- [Connecting the Dots: Multivariate Time Series Forecasting with Graph Neural Networks](https://mlsanity.com/paper/2005.11650/) (2020)
- [LLM2Vec: Large Language Models Are Secretly Powerful Text Encoders](https://mlsanity.com/paper/2404.05961/) (2024)
- [Fine-Tuning LLaMA for Multi-Stage Text Retrieval](https://mlsanity.com/paper/2310.08319/) (2023)
- [Whitening Sentence Representations for Better Semantics and Faster Retrieval](https://mlsanity.com/paper/2103.15316/) (2021)
- [Deep Time Series Models: A Comprehensive Survey and Benchmark](https://mlsanity.com/paper/2407.13278/) (2024)
- [TEST: Text Prototype Aligned Embedding to Activate LLM's Ability for Time Series](https://mlsanity.com/paper/2308.08241/) (2023)
- [Self-supervised Learning in Remote Sensing: A Review](https://mlsanity.com/paper/2206.13188/) (2022)
- [LLM4TS: Aligning Pre-Trained LLMs as Data-Efficient Time-Series Forecasters](https://mlsanity.com/paper/2308.08469/) (2023)
- [LargeST: A Benchmark Dataset for Large-Scale Traffic Forecasting](https://mlsanity.com/paper/2306.08259/) (2023)
- [Tile2Vec: Unsupervised representation learning for spatially distributed data](https://mlsanity.com/paper/1805.02855/) (2018)
- [Large Language Models are Geographically Biased](https://mlsanity.com/paper/2402.02680/) (2024)
- [On the Opportunities and Challenges of Foundation Models for Geospatial Artificial Intelligence](https://mlsanity.com/paper/2304.06798/) (2023)
- [UniST: A Prompt-Empowered Universal Model for Urban Spatio-Temporal Prediction](https://mlsanity.com/paper/2402.11838/) (2024)
- [Multi-Range Attentive Bicomponent Graph Convolutional Network for Traffic Forecasting](https://mlsanity.com/paper/1911.12093/) (2019)
- [K2: A Foundation Language Model for Geoscience Knowledge Understanding and Utilization](https://mlsanity.com/paper/2306.05064/) (2023)
- [Empowering Time Series Analysis with Large Language Models: A Survey](https://mlsanity.com/paper/2402.03182/) (2024)
- [SustainBench: Benchmarks for Monitoring the Sustainable Development Goals with Machine Learning](https://mlsanity.com/paper/2111.04724/) (2021)
- [ImputeFormer: Low Rankness-Induced Transformers for Generalizable Spatiotemporal Imputation](https://mlsanity.com/paper/2312.01728/) (2023)
- [Urban2Vec: Incorporating Street View Imagery and POIs for Multi-Modal Urban Neighborhood Embedding](https://mlsanity.com/paper/2001.11101/) (2020)
- [Spatial-Temporal Hypergraph Self-Supervised Learning for Crime Prediction](https://mlsanity.com/paper/2204.08587/) (2022)
- [Domain Adversarial Spatial-Temporal Network: A Transferable Framework for Short-term Traffic Forecasting across Cities](https://mlsanity.com/paper/2202.03630/) (2022)
- [Taming Local Effects in Graph-based Spatiotemporal Forecasting](https://mlsanity.com/paper/2302.04071/) (2023)

## Built on by (most cited first, 15 of 15)

- [Exploring the Roles of Large Language Models in Reshaping Transportation Systems: A Survey, Framework, and Roadmap](https://mlsanity.com/paper/2503.21411/) (2025)
- [LLM-attacker: Enhancing Closed-loop Adversarial Scenario Generation for Autonomous Driving with Large Language Models](https://mlsanity.com/paper/2501.15850/) (2025)
- [Ultra-High Resolution Segmentation via Boundary-Enhanced Patch-Merging Transformer](https://mlsanity.com/paper/2412.10181/) (2024)
- [CrashSage: A Large Language Model-Centered Framework for Contextual and Interpretable Traffic Crash Analysis](https://mlsanity.com/paper/2505.07853/) (2025)
- [Learning to Factorize and Adapt: A Versatile Approach Toward Universal Spatio-Temporal Foundation Models](https://mlsanity.com/paper/2601.12083/) (2026)
- [Virtual Traffic Police: Large Language Model-Augmented Traffic Signal Control for Unforeseen Incidents](https://mlsanity.com/paper/2601.15816/) (2026)
- [STeInFormer: Spatial-Temporal Interaction Transformer Architecture for Remote Sensing Change Detection](https://mlsanity.com/paper/2412.17247/) (2024)
- [Coordinates from Context: Using LLMs to Ground Complex Location References](https://mlsanity.com/paper/2510.08741/) (2025)
- [Comprehending Spatio-temporal Data via Cinematic Storytelling using Large Language Models](https://mlsanity.com/paper/2510.17301/) (2025)
- [AskNearby: An LLM-Based Application for Neighborhood Information Retrieval and Personalized Cognitive-Map Recommendations](https://mlsanity.com/paper/2512.02502/) (2025)
- [OsmT: Bridging OpenStreetMap Queries and Natural Language with Open-source Tag-aware Language Models](https://mlsanity.com/paper/2512.04738/) (2025)
- [Bridge: Retrieval-Augmented Spatiotemporal Modeling for Urban Delivery Demand](https://mlsanity.com/paper/2605.19172/) (2026)
- [Textual Supervision Enhances Geospatial Representations in Vision-Language Models](https://mlsanity.com/paper/2606.07172/) (2026)
- [General Semantic Knowledge Infusion for Spatio-Temporal Traffic Forecasting](https://mlsanity.com/paper/2608.17440/) (2026)
- [On the security and privacy of LLMs in Mobility](https://mlsanity.com/paper/2609.26295/) (2026)

## Abstract (arXiv)

In the geospatial domain, universal representation models are significantly less prevalent than their extensive use in natural language processing and computer vision. This discrepancy arises primarily from the high costs associated with the input of existing representation models, which often require street views and mobility data. To address this, we develop a novel, training-free method that leverages large language models (LLMs) and auxiliary map data from OpenStreetMap to derive geolocation representations (LLMGeovec). LLMGeovec can represent the geographic semantics of city, country, and global scales, which acts as a generic enhancer for spatio-temporal learning. Specifically, by direct feature concatenation, we introduce a simple yet effective paradigm for enhancing multiple spatio-temporal tasks including geographic prediction (GP), long-term time series forecasting (LTSF), and graph-based spatio-temporal forecasting (GSTF). LLMGeovec can seamlessly integrate into a wide spectrum of spatio-temporal learning models, providing immediate enhancements. Experimental results demonstrate that LLMGeovec achieves global coverage and significantly boosts the performance of leading GP, LTSF, and GSTF models. Our codes are available at urlhttps://github.com/Umaruchain/LLMGeovec.