# Protocol before progress: leakage-aware evaluation of AIS trajectory prediction

arXiv: https://arxiv.org/abs/2609.25827 · HTML: https://mlsanity.com/paper/2609.25827/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 18 of its 32 references that are in the map.
- First posted on arXiv 22 Sep 2026.

## Builds on (most cited first, 18 of 18)

- [A Time Series is Worth 64 Words: Long-term Forecasting with Transformers](https://mlsanity.com/paper/2211.14730/) (2022)
- [FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecasting](https://mlsanity.com/paper/2201.12740/) (2022)
- [Deep Reinforcement Learning that Matters](https://mlsanity.com/paper/1709.06560/) (2017)
- [Argoverse: 3D Tracking and Forecasting with Rich Maps](https://mlsanity.com/paper/1911.02620/) (2019)
- [Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networks](https://mlsanity.com/paper/1803.10892/) (2018)
- [CSDI: Conditional Score-based Diffusion Models for Probabilistic Time Series Imputation](https://mlsanity.com/paper/2107.03502/) (2021)
- [Recurrent Neural Networks for Multivariate Time Series with Missing Values](https://mlsanity.com/paper/1606.01865/) (2016)
- [A Metric Learning Reality Check](https://mlsanity.com/paper/2003.08505/) (2020)
- [Are We Really Making Much Progress? A Worrying Analysis of Recent Neural Recommendation Approaches](https://mlsanity.com/paper/1907.06902/) (2019)
- [SAITS: Self-Attention-based Imputation for Time Series](https://mlsanity.com/paper/2202.08516/) (2022)
- [On the State of the Art of Evaluation in Neural Language Models](https://mlsanity.com/paper/1707.05589/) (2017)
- [Evaluating time series forecasting models: An empirical study on performance estimation methods](https://mlsanity.com/paper/1905.11744/) (2019)
- [Deep Learning Methods for Vessel Trajectory Prediction based on Recurrent Neural Networks](https://mlsanity.com/paper/2101.02486/) (2021)
- [Analyzing the Variety Loss in the Context of Probabilistic Trajectory Prediction](https://mlsanity.com/paper/1907.10178/) (2019)
- [AIS Data-Driven Maritime Monitoring Based on Transformer: A Comprehensive Review](https://mlsanity.com/paper/2505.07374/) (2025)
- [DiffuTraj: A Stochastic Vessel Trajectory Prediction Approach via Guided Diffusion Process](https://mlsanity.com/paper/2410.09550/) (2024)
- [STGDPM:Vessel Trajectory Prediction with Spatio-Temporal Graph Diffusion Probabilistic Model](https://mlsanity.com/paper/2503.08065/) (2025)
- [EnvShip: A Unified Framework for Context-Aware and Cross-Region Vessel Trajectory Forecasting](https://mlsanity.com/paper/2606.15240/) (2026)

## Abstract (arXiv)

Reported gains in vessel-trajectory prediction from Automatic Identification System (AIS) data are credited to new architectures, but the evaluation protocol is rarely measured as a source of error reduction. We build a leakage-aware protocol with vessel-, time- and region-disjoint splits and apply it to two corpora with different traffic: 31 days of Danish national AIS traffic and 30 days of US Gulf coast traffic off Houston and Galveston. On both, we audit TrAISformer, GATransformer, and controlled AISFormer-inspired reconstructions. Three protocol effects appear in both corpora. First, TrAISformer's best-of-16 oracle decoder lowers error by a factor of 2.1-3.2 relative to greedy decoding. Second, a split that shares vessels lowers its greedy error by 23-25% at one hour, against 2% or less for a compact 0.43 M-parameter encoder. Third, a region-disjoint split raises TrAISformer's one-hour error from 2.2 to 24.6 km on the US corpus, because 99.9% of the test contexts fall in longitude bins never seen in training; the encoder built on local offsets is unaffected by this. Architectural mechanisms matter less: GATransformer's graph attention gives no measurable benefit on either corpus, while its waterway feature is worth 12-22%. The effect of a time-disjoint split is not stable across corpora (13% versus 2%). We release the splits and code.