# Leveraging Language Foundation Models for Human Mobility Forecasting

arXiv: https://arxiv.org/abs/2209.05479 · HTML: https://mlsanity.com/paper/2209.05479/

## Key facts

- Cited by 50 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 24 references that are in the map.
- First posted on arXiv 11 Sep 2022.

## Builds on (most cited first, 17 of 17)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019)
- [Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling](https://mlsanity.com/paper/1412.3555/) (2014)
- [Learning to Prompt for Vision-Language Models](https://mlsanity.com/paper/2109.01134/) (2021)
- [Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting](https://mlsanity.com/paper/2012.07436/) (2020)
- [Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting](https://mlsanity.com/paper/2106.13008/) (2021)
- [Reformer: The Efficient Transformer](https://mlsanity.com/paper/2001.04451/) (2020)
- [Enhancing the Locality and Breaking the Memory Bottleneck of Transformer on Time Series Forecasting](https://mlsanity.com/paper/1907.00235/) (2019)
- [MobTCast: Leveraging Auxiliary Trajectory Forecasting for Human Mobility Prediction](https://mlsanity.com/paper/2110.01401/) (2021)
- [Numeracy for Language Models: Evaluating and Improving their Ability to Predict Numbers](https://mlsanity.com/paper/1805.08154/) (2018)
- [An Empirical Investigation of Contextualized Number Prediction](https://mlsanity.com/paper/2011.07961/) (2020)
- [TERMCast: Temporal Relation Modeling for Effective Urban Flow Forecasting](https://mlsanity.com/paper/2011.05554/) (2020)

## Built on by (most cited first, 30 of 50)

- [Foundation Models for Time Series Analysis: A Tutorial and Survey](https://mlsanity.com/paper/2403.14735/) (2024)
- [Large Language Models for Time Series: A Survey](https://mlsanity.com/paper/2402.01801/) (2024)
- [Large Models for Time Series and Spatio-Temporal Data: A Survey and Outlook](https://mlsanity.com/paper/2310.10196/) (2023)
- [TimeCMA: Towards LLM-Empowered Multivariate Time Series Forecasting via Cross-Modality Alignment](https://mlsanity.com/paper/2406.01638/) (2024)
- [Spatial-Temporal Large Language Model for Traffic Prediction](https://mlsanity.com/paper/2401.10134/) (2024)
- [Empowering Time Series Analysis with Large Language Models: A Survey](https://mlsanity.com/paper/2402.03182/) (2024)
- [Large Language Models for Next Point-of-Interest Recommendation](https://mlsanity.com/paper/2404.17591/) (2024)
- [ITINERA: Integrating Spatial Optimization with Large Language Models for Open-domain Urban Itinerary Planning](https://mlsanity.com/paper/2402.07204/) (2024)
- [Deep Learning for Cross-Domain Data Fusion in Urban Computing: Taxonomy, Advances, and Outlook](https://mlsanity.com/paper/2402.19348/) (2024)
- [Can Language Beat Numerical Regression? Language-Based Multimodal Trajectory Prediction](https://mlsanity.com/paper/2403.18447/) (2024)
- [Large Language Models for Spatial Trajectory Patterns Mining](https://mlsanity.com/paper/2310.04942/) (2023)
- [Exploring the Roles of Large Language Models in Reshaping Transportation Systems: A Survey, Framework, and Roadmap](https://mlsanity.com/paper/2503.21411/) (2025)
- [Foundation Models for Spatio-Temporal Data Science: A Tutorial and Survey](https://mlsanity.com/paper/2503.13502/) (2025)
- [Towards Urban General Intelligence: A Review and Outlook of Urban Foundation Models](https://mlsanity.com/paper/2402.01749/) (2024)
- [Empowering Time Series Analysis with Foundation Models: A Comprehensive Survey](https://mlsanity.com/paper/2405.02358/) (2024)
- [Large Language Models are Zero-Shot Next Location Predictors](https://mlsanity.com/paper/2405.20962/) (2024)
- [Towards Modeling Learner Performance with Large Language Models](https://mlsanity.com/paper/2403.14661/) (2024)
- [How Can Time Series Analysis Benefit From Multiple Modalities? A Survey and Outlook](https://mlsanity.com/paper/2503.11835/) (2025)
- [Evaluating Large Language Models as Virtual Annotators for Time-series Physical Sensing Data](https://mlsanity.com/paper/2403.01133/) (2024)
- [Large Language Models for Mobility Analysis in Transportation Systems: A Survey on Forecasting Tasks](https://mlsanity.com/paper/2405.02357/) (2024)
- [BIGCity: A Universal Spatiotemporal Model for Unified Trajectory and Traffic State Data Analysis](https://mlsanity.com/paper/2412.00953/) (2024)
- [Large Language Models for Travel Behavior Prediction](https://mlsanity.com/paper/2312.00819/) (2023)
- [Tri-Level Navigator: LLM-Empowered Tri-Level Learning for Time Series OOD Generalization](https://mlsanity.com/paper/2410.07018/) (2024)
- [A Survey on Mathematical Reasoning and Optimization with Large Language Models](https://mlsanity.com/paper/2503.17726/) (2025)
- [Humans vs Large Language Models: Judgmental Forecasting in an Era of Advanced AI](https://mlsanity.com/paper/2312.06941/) (2023)
- [Deep Learning for Time Series Forecasting: A Survey](https://mlsanity.com/paper/2503.10198/) (2025)
- [Unraveling Spatio-Temporal Foundation Models via the Pipeline Lens: A Comprehensive Review](https://mlsanity.com/paper/2506.01364/) (2025)
- [Utilizing Language Models for Energy Load Forecasting](https://mlsanity.com/paper/2310.17788/) (2023)
- [MobilityDL: A Review of Deep Learning From Trajectory Data](https://mlsanity.com/paper/2402.00732/) (2024)
- [Trajectory Anomaly Detection with Language Models](https://mlsanity.com/paper/2409.15366/) (2024)

## Abstract (arXiv)

In this paper, we propose a novel pipeline that leverages language foundation models for temporal sequential pattern mining, such as for human mobility forecasting tasks. For example, in the task of predicting Place-of-Interest (POI) customer flows, typically the number of visits is extracted from historical logs, and only the numerical data are used to predict visitor flows. In this research, we perform the forecasting task directly on the natural language input that includes all kinds of information such as numerical values and contextual semantic information. Specific prompts are introduced to transform numerical temporal sequences into sentences so that existing language models can be directly applied. We design an AuxMobLCast pipeline for predicting the number of visitors in each POI, integrating an auxiliary POI category classification task with the encoder-decoder architecture. This research provides empirical evidence of the effectiveness of the proposed AuxMobLCast pipeline to discover sequential patterns in mobility forecasting tasks. The results, evaluated on three real-world datasets, demonstrate that pre-trained language foundation models also have good performance in forecasting temporal sequences. This study could provide visionary insights and lead to new research directions for predicting human mobility.