mlsanityData to 14 Oct 2026

Efficient Temporal Tokenization for Mobility Prediction with Large Language Models

arXiv 2507.14017 ↗First posted 18 Jul 2025cs.CL cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

We introduce RHYTHM (Reasoning with Hierarchical Temporal Tokenization for Human Mobility), a framework that leverages large language models (LLMs) as spatio-temporal predictors and trajectory reasoners. RHYTHM partitions trajectories into daily segments encoded as discrete tokens with hierarchical attention, capturing both daily and weekly dependencies while substantially reducing the sequence length. Token representations are enriched with pre-computed prompt embeddings via a frozen LLM, enhancing the model's ability to capture interdependencies without extensive computational overhead. By freezing the LLM backbone, RHYTHM achieves significant computational efficiency. Evaluation on three real-world datasets demonstrates a 2.4% improvement in accuracy, 5.0% increase on weekends, and 24.6% reduction in training time compared to state-of-the-art methods.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 37 of its 62 references that are in the map. First posted on arXiv 18 Jul 2025.

Builds on

37 of 62 references.

References · 37

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  5. 2024The Llama 3 Herd of Modelscited by 14,715
  6. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  7. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  8. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
  9. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
  10. 2022A Time Series is Worth 64 Words: Long-term Forecasting with Transformerscited by 2,039
  11. 2019BioBERT: a pre-trained biomedical language representation model for biomedical text miningcited by 1,947
  12. 2022Large Language Models Encode Clinical Knowledgecited by 1,830
25 more
  1. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,386
  2. 2023BloombergGPT: A Large Language Model for Financecited by 939
  3. 2023iTransformer: Inverted Transformers Are Effective for Time Series Forecastingcited by 931
  4. 2023Time-LLM: Time Series Forecasting by Reprogramming Large Language Modelscited by 700
  5. 2021Multimodal Few-Shot Learning with Frozen Language Modelscited by 661
  6. 2023One Fits All:Power General Time Series Analysis by Pretrained LMcited by 632
  7. 2022BioGPT: Generative Pre-trained Transformer for Biomedical Text Generation and Miningcited by 538
  8. 2020Hopfield Networks is All You Needcited by 475
  9. 2022Revisiting Neural Scaling Laws in Language and Visioncited by 132
  10. 2023LLM4TS: Aligning Pre-Trained LLMs as Data-Efficient Time-Series Forecasterscited by 129
  11. 2021STAN: Spatio-Temporal Attention Network for Next Location Recommendationcited by 102
  12. 2023Where Would I Go Next? Large Language Models as Human Mobility Predictorscited by 93
  13. 2023GETNext: Trajectory Flow Map Enhanced Transformer for Next POI Recommendationcited by 58
  14. 2024Outlier-Efficient Hopfield Layers for Large Transformer-Based Modelscited by 38
  15. 2024Computational Limits of Low-Rank Adaptation (LoRA) Fine-Tuning for Transformer Modelscited by 31
  16. 2024COLA: Cross-city Mobility Transformer for Human Trajectory Simulationcited by 25
  17. 2024AgentMove: A Large Language Model based Agentic Framework for Zero-shot Next Location Predictioncited by 25
  18. 2024Mobility-LLM: Learning Visiting Intentions and Travel Preferences from Human Mobility Data with Large Language Modelscited by 22
  19. 2022Context-aware multi-head self-attentional neural network model for next location predictioncited by 21
  20. 2024Conv-CoA: Improving Open-domain Question Answering in Large Language Models via Conversational Chain-of-Actioncited by 19
  21. 2024TrajGPT: Controlled Synthetic Trajectory Generation Using a Multitask Transformer-Based Spatiotemporal Modelcited by 13
  22. 2023Human mobility is well described by closed-form gravity-like models learned automatically from datacited by 11
  23. 2024Pretrained Mobility Transformer: A Foundation Model for Human Mobilitycited by 8
  24. 2025Fast and Low-Cost Genomic Foundation Models via Outlier Removalcited by 8
  25. 2024ST-MoE-BERT: A Spatial-Temporal Mixture-of-Experts Framework for Long-Term Cross-City Mobility Predictioncited by 7

Built on by

Not cited yet.