# TrAISformer -- A Transformer Network with Sparse Augmented Data Representation and Cross Entropy Loss for AIS-based Vessel Trajectory Prediction

arXiv: https://arxiv.org/abs/2109.03958 · HTML: https://mlsanity.com/paper/2109.03958/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 78 references that are in the map.
- First posted on arXiv 8 Sep 2021.

## Builds on (most cited first, 23 of 23)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Variational Inference with Normalizing Flows](https://mlsanity.com/paper/1505.05770/) (2015)
- [Knowledge Distillation: A Survey](https://mlsanity.com/paper/2006.05525/) (2020)
- [Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI](https://mlsanity.com/paper/1910.10045/) (2019)
- [A Convergence Theory for Deep Learning via Over-Parameterization](https://mlsanity.com/paper/1811.03962/) (2018)
- [Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networks](https://mlsanity.com/paper/1803.10892/) (2018)
- [NVAE: A Deep Hierarchical Variational Autoencoder](https://mlsanity.com/paper/2007.03898/) (2020)
- [Importance Estimation for Neural Network Pruning](https://mlsanity.com/paper/1906.10771/) (2019)
- [A Recurrent Latent Variable Model for Sequential Data](https://mlsanity.com/paper/1506.02216/) (2015)
- [TNT: Target-driveN Trajectory Prediction](https://mlsanity.com/paper/2008.08294/) (2020)
- [SoPhie: An Attentive GAN for Predicting Paths Compliant to Social and Physical Constraints](https://mlsanity.com/paper/1806.01482/) (2018)
- [DenseTNT: End-to-end Trajectory Prediction from Dense Goal Sets](https://mlsanity.com/paper/2108.09640/) (2021)
- [On the Optimization of Deep Networks: Implicit Acceleration by Overparameterization](https://mlsanity.com/paper/1802.06509/) (2018)
- [Human Motion Trajectory Prediction: A Survey](https://mlsanity.com/paper/1905.06113/) (2019)
- [Social Attention: Modeling Attention in Human Crowds](https://mlsanity.com/paper/1710.04689/) (2017)
- [Variational Autoencoder for Deep Learning of Images, Labels and Captions](https://mlsanity.com/paper/1609.08976/) (2016)
- [Deep Learning Methods for Vessel Trajectory Prediction based on Recurrent Neural Networks](https://mlsanity.com/paper/2101.02486/) (2021)
- [GeoTrackNet-A Maritime Anomaly Detector using Probabilistic Neural Network Representation of AIS Tracks and A Contrario Detection](https://mlsanity.com/paper/1912.00682/) (2019)
- [A Multi-task Deep Learning Architecture for Maritime Surveillance using AIS Data Streams](https://mlsanity.com/paper/1806.03972/) (2018)
- [Space-based Global Maritime Surveillance. Part II: Artificial Intelligence and Data Fusion Techniques](https://mlsanity.com/paper/2011.11338/) (2020)
- [Detection of Abnormal Vessel Behaviours from AIS data using GeoTrackNet: from the Laboratory to the Ocean](https://mlsanity.com/paper/2008.05443/) (2020)

## Built on by (most cited first, 20 of 20)

- [Multi-Path Long-Term Vessel Trajectories Forecasting with Probabilistic Feature Fusion for Problem Shifting](https://mlsanity.com/paper/2310.18948/) (2023)
- [AIS Data-Driven Maritime Monitoring Based on Transformer: A Comprehensive Review](https://mlsanity.com/paper/2505.07374/) (2025)
- [Tensor-Var: Efficient Four-Dimensional Variational Data Assimilation](https://mlsanity.com/paper/2501.13312/) (2025)
- [AIS-LLM: A Unified Framework for Maritime Trajectory Prediction, Anomaly Detection, and Collision Risk Assessment with Explainable Forecasting](https://mlsanity.com/paper/2508.07668/) (2025)
- [Enhancing Maritime Trajectory Forecasting via H3 Index and Causal Language Modelling (CLM)](https://mlsanity.com/paper/2405.09596/) (2024)
- [Physics-Informed Neural Networks for Vessel Trajectory Prediction: Learning Time-Discretized Kinematic Dynamics via Finite Differences](https://mlsanity.com/paper/2506.12029/) (2025)
- [Incorporating Navigation Context into Inland Vessel Trajectory Prediction: A Gaussian Mixture Model and Transformer Approach](https://mlsanity.com/paper/2406.02344/) (2024)
- [Goal-Conditioned Reinforcement Learning for Data-Driven Maritime Navigation](https://mlsanity.com/paper/2509.01838/) (2025)
- [SKETCH: Semantic Key-Point Conditioning for Long-Horizon Vessel Trajectory Prediction](https://mlsanity.com/paper/2601.18537/) (2026)
- [CmIVTP: Cross-modal Interaction-based Vessel Trajectory Prediction for Maritime Intelligence](https://mlsanity.com/paper/2605.26524/) (2026)
- [Multi-vessel Interaction-Aware Trajectory Prediction and Collision Risk Assessment](https://mlsanity.com/paper/2509.01836/) (2025)
- [An Automated Tip-and-Cue Framework for Optimized Satellite Tasking and Visual Intelligence](https://mlsanity.com/paper/2512.09670/) (2025)
- [Predictive Modeling of Maritime Radar Data Using Transformer Architecture](https://mlsanity.com/paper/2512.17098/) (2025)
- [Spatio-Temporal Graphs Beyond Grids: Benchmark for Maritime Anomaly Detection](https://mlsanity.com/paper/2512.20086/) (2025)
- [Hierarchical Two-Stage Framework for Environment-Aware Long-Horizon Vessel Trajectory Prediction](https://mlsanity.com/paper/2605.16442/) (2026)
- [M-CTX: Exact and Scalable Spatial Context Retrieval for Trajectory Analytics](https://mlsanity.com/paper/2606.15244/) (2026)
- [NaviAIS: A Scenario-Level Vessel Trajectory Prediction Dataset withVectorized Lane Priors and the NaviLane Forecasting Framework](https://mlsanity.com/paper/2607.18887/) (2026)
- [Does Latent Context Help? A Controlled Evaluation of Inverse Reinforcement Learning in Arctic Shipping](https://mlsanity.com/paper/2608.06105/) (2026)
- [Non-Parametric Spatiotemporal Trajectory Prediction via State-Conditioned Transition Sampling](https://mlsanity.com/paper/2608.14349/) (2026)
- [M3-Former: Multimodal Transformer with Mixture-of-Experts for Long-Term Vessel Trajectory Prediction](https://mlsanity.com/paper/2609.10559/) (2026)

## Abstract (arXiv)

Vessel trajectory prediction plays a pivotal role in numerous maritime applications and services. While the Automatic Identification System (AIS) offers a rich source of information to address this task, forecasting vessel trajectory using AIS data remains challenging, even for modern machine learning techniques, because of the inherent heterogeneous and multimodal nature of motion data. In this paper, we propose a novel approach to tackle these challenges. We introduce a discrete, high-dimensional representation of AIS data and a new loss function designed to explicitly address heterogeneity and multimodality. The proposed model-referred to as TrAISformer-is a modified transformer network that extracts long-term temporal patterns in AIS vessel trajectories in the proposed enriched space to forecast the positions of vessels several hours ahead. We report experimental results on real, publicly available AIS data. TrAISformer significantly outperforms state-of-the-art methods, with an average prediction performance below 10 nautical miles up to ~10 hours.