mlsanityData to 7 Oct 2026

TraXion: Rethinking Pre-training Frameworks for Mobility and Beyond

arXiv 2605.06906 ↗First posted 7 May 2026cs.LGcited by 0

On the map: LLM reasoning → Spatial and embodied reasoning

Abstract (arXiv)

Human mobility differs from text and from generic time series in three structural ways: visits are tuple-valued events whose meaning depends on the joint distribution over location, time, and activity; users carry persistent signatures across trajectories; and visits are not independent across users, since co-location at shared places is a primary signal. Existing pre-training recipes for mobility import objectives from language modeling, treating trajectories as sentences and visits as tokens, an analogy that fails against each of the three properties above. These properties define a broader class, multi-entity spatiotemporal event streams (MESES), spanning enterprise authentication logs, electronic health records, and other event-stream domains where entities share infrastructure, schedules, or contexts. We make the properties precise as three axioms that any pre-training framework for MESES should satisfy, and introduce TraXion, whose objectives and architecture are jointly designed to meet them. A single TraXion checkpoint per dataset beats task-specific baselines on every task across six public mobility datasets covering anomaly detection, next-POI recommendation, next-visit prediction, and social-link prediction. The same recipe, applied unchanged to enterprise authentication logs and ICU mortality prediction, matches or exceeds prior work on both, showing that event streams from domains as different as mobility, security, and healthcare can be modeled under a single framework.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning. Builds on 53 of its 77 references that are in the map. First posted on arXiv 7 May 2026.

Builds on

53 of 77 references.

References · 53

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  3. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  4. 2022A Time Series is Worth 64 Words: Long-term Forecasting with Transformerscited by 1,981
  5. 2019Similarity of Neural Network Representations Revisitedcited by 1,695
  6. 2018Self-Attentive Sequential Recommendationcited by 1,169
  7. 2019BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformercited by 952
  8. 2019Temporal Fusion Transformers for Interpretable Multi-horizon Time Series Forecastingcited by 748
  9. 2020Prototypical Contrastive Learning of Unsupervised Representationscited by 636
  10. 2023One Fits All:Power General Time Series Analysis by Pretrained LMcited by 615
  11. 2016Recurrent Neural Networks for Multivariate Time Series with Missing Valuescited by 548
  12. 2021TS2Vec: Towards Universal Representation of Time Seriescited by 463
38 more
  1. 2020A Transformer-based Framework for Multivariate Time Series Representation Learningcited by 449
  2. 2020Med-BERT: pre-trained contextualized embeddings on large-scale structured electronic health records for disease predictioncited by 368
  3. 2020S^3-Rec: Self-Supervised Learning for Sequential Recommendation with Mutual Information Maximizationcited by 311
  4. 2019BEHRT: Transformer for Electronic Health Recordscited by 258
  5. 2021Multi-Time Attention Networks for Irregularly Sampled Time Seriescited by 234
  6. 2019Time2Vec: Learning a Vector Representation of Timecited by 208
  7. 2021LogBERT: Log Anomaly Detection via BERTcited by 114
  8. 2021Neural Temporal Point Processes: A Reviewcited by 113
  9. 2023SimMTM: A Simple Pre-Training Framework for Masked Time-Series Modelingcited by 112
  10. 2021Graph-Guided Network for Irregularly Sampled Multivariate Time Seriescited by 110
  11. 2021Self-Supervised Transformer for Sparse and Irregularly Sampled Multivariate Clinical Time-Seriescited by 96
  12. 2021Transformer Embeddings of Irregularly Spaced Events and Their Participantscited by 90
  13. 2020Multi-Scale Representation Learning for Spatial Feature Distributions using Grid Cellscited by 88
  14. 2024UniST: A Prompt-Empowered Universal Model for Urban Spatio-Temporal Predictioncited by 86
  15. 2021CEHR-BERT: Incorporating temporal information from structured EHR data to improve prediction taskscited by 75
  16. 2024Zero Shot Health Trajectory Prediction Using Transformercited by 74
  17. 2023TabR: Tabular Deep Learning Meets Nearest Neighbors in 2023cited by 73
  18. 2023GETNext: Trajectory Flow Map Enhanced Transformer for Next POI Recommendationcited by 57
  19. 2021MobTCast: Leveraging Auxiliary Trajectory Forecasting for Human Mobility Predictioncited by 48
  20. 2022Contrastive Trajectory Similarity Learning with Dual-Feature Attentioncited by 44
  21. 2014Discovering Latent Network Structure in Point Process Datacited by 28
  22. 2023LogGPT: Log Anomaly Detection via GPTcited by 26
  23. 2024AgentMove: A Large Language Model based Agentic Framework for Zero-shot Next Location Predictioncited by 25
  24. 2020Self-Supervised Log Parsingcited by 23
  25. 2024Mobility-LLM: Learning Visiting Intentions and Travel Preferences from Human Mobility Data with Large Language Modelscited by 21
  26. 2017Deep Learning for Unsupervised Insider Threat Detection in Structured Cybersecurity Data Streamscited by 19
  27. 2024UniTraj: Learning a Universal Trajectory Foundation Model from Billion-Scale Worldwide Tracescited by 18
  28. 2022Mutual Distillation Learning Network for Trajectory-User Linkingcited by 15
  29. 2024TrajGPT: Controlled Synthetic Trajectory Generation Using a Multitask Transformer-Based Spatiotemporal Modelcited by 13
  30. 2024NUMOSIM: A Synthetic Mobility Dataset with Anomaly Detection Benchmarkscited by 12
  31. 2022Improving Event Representation via Simultaneous Weakly Supervised Contrastive Learning and Clusteringcited by 11
  32. 2024MobilityGPT: Enhanced Human Mobility Modeling with a GPT modelcited by 11
  33. 2024Urban Anomalies: A Simulated Human Mobility Dataset with Injected Anomaliescited by 10
  34. 2024Taming the Long Tail in Human Mobility Predictioncited by 8
  35. 2023Trajectory-User Linking via Hierarchical Spatio-Temporal Attention Networkscited by 7
  36. 2025CoBAD: Modeling Collective Behaviors for Human Mobility Anomaly Detectioncited by 5
  37. 2026TrajFlow: Nation-wide Pseudo GPS Trajectory Generation with Flow Matching Modelscited by 4
  38. 2026Mobility-Embedded POIs: Learning What A Place Is and How It Is Used from Human Movementcited by 3

50 most cited of 53 All 53 (JSON)

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026MobiDiff: Semantic-Aware Multi-Channel Discrete Diffusion for Human Mobility Data Generation
  2. 2025MoveGPT: Scaling Mobility Foundation Models with Spatially-Aware Mixture of Experts
  3. 2025SeMob: Semantic Synthesis for Dynamic Urban Mobility Prediction
  4. 2025RHYTHM: Reasoning with Hierarchical Temporal Tokenization for Human Mobility
  5. 2024A Universal Model for Human Mobility Prediction
  6. 2026ARMove: Learning to Predict Human Mobility through Agentic Reasoning
  7. 2024LIMP: Large Language Model Enhanced Intent-aware Mobility Prediction
  8. 2025MoE-TransMov: A Transformer-based Model for Next POI Prediction in Familiar & Unfamiliar Movements