# STEP: Structured Training and Evaluation Platform for benchmarking trajectory prediction models

arXiv: https://arxiv.org/abs/2509.14801 · HTML: https://mlsanity.com/paper/2509.14801/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 43 references that are in the map.
- First posted on arXiv 18 Sep 2025.

## Builds on (most cited first, 30 of 32)

- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [Scalability in Perception for Autonomous Driving: Waymo Open Dataset](https://mlsanity.com/paper/1912.04838/) (2019)
- [Argoverse 2: Next Generation Datasets for Self-Driving Perception and Forecasting](https://mlsanity.com/paper/2301.00493/) (2023)
- [Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networks](https://mlsanity.com/paper/1803.10892/) (2018)
- [Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Dataset](https://mlsanity.com/paper/2104.10133/) (2021)
- [AgentFormer: Agent-Aware Transformers for Socio-Temporal Multi-Agent Forecasting](https://mlsanity.com/paper/2103.14023/) (2021)
- [Wayformer: Motion Forecasting via Simple & Efficient Attention Networks](https://mlsanity.com/paper/2207.05844/) (2022)
- [The highD Dataset: A Drone Dataset of Naturalistic Vehicle Trajectories on German Highways for Validation of Highly Automated Driving Systems](https://mlsanity.com/paper/1810.05642/) (2018)
- [Motion Transformer with Global Intention Localization and Local Movement Refinement](https://mlsanity.com/paper/2209.13508/) (2022)
- [One Thousand and One Hours: Self-driving Motion Prediction Dataset](https://mlsanity.com/paper/2006.14480/) (2020)
- [INTERACTION Dataset: An INTERnational, Adversarial and Cooperative moTION Dataset in Interactive Driving Scenarios with Semantic Maps](https://mlsanity.com/paper/1910.03088/) (2019)
- [MTR++: Multi-Agent Motion Prediction with Symmetric Scene Modeling and Guided Intention Querying](https://mlsanity.com/paper/2306.17770/) (2023)
- [The inD Dataset: A Drone Dataset of Naturalistic Road User Trajectories at German Intersections](https://mlsanity.com/paper/1911.07602/) (2019)
- [Latent Variable Sequential Set Transformers For Joint Multi-Agent Motion Prediction](https://mlsanity.com/paper/2104.00563/) (2021)
- [Human Trajectory Forecasting in Crowds: A Deep Learning Perspective](https://mlsanity.com/paper/2007.03639/) (2020)
- [On Adversarial Robustness of Trajectory Prediction for Autonomous Vehicles](https://mlsanity.com/paper/2201.05057/) (2022)
- [UniTraj: A Unified Framework for Scalable Vehicle Trajectory Prediction](https://mlsanity.com/paper/2403.15098/) (2024)
- [AdvDO: Realistic Adversarial Attacks for Trajectory Prediction](https://mlsanity.com/paper/2209.08744/) (2022)
- [Graph Neural Networks for Modelling Traffic Participant Interaction](https://mlsanity.com/paper/1903.01254/) (2019)
- [How many Observations are Enough? Knowledge Distillation for Trajectory Forecasting](https://mlsanity.com/paper/2203.04781/) (2022)
- [Analyzing the Variety Loss in the Context of Probabilistic Trajectory Prediction](https://mlsanity.com/paper/1907.10178/) (2019)
- [trajdata: A Unified Interface to Multiple Human Trajectory Datasets](https://mlsanity.com/paper/2307.13924/) (2023)
- [Adapting to Length Shift: FlexiLength Network for Trajectory Prediction](https://mlsanity.com/paper/2404.00742/) (2024)
- [FloMo: Tractable Motion Prediction with Normalizing Flows](https://mlsanity.com/paper/2103.03614/) (2021)
- [Targeted Adversarial Attacks against Neural Network Trajectory Predictors](https://mlsanity.com/paper/2212.04138/) (2022)
- [The Atlas Benchmark: an Automated Evaluation Framework for Human Motion Prediction](https://mlsanity.com/paper/2207.09830/) (2022)
- [Trajectory Prediction with Observations of Variable-Length for Motion Planning in Highway Merging scenarios](https://mlsanity.com/paper/2306.05478/) (2023)
- [Benchmark for Models Predicting Human Behavior in Gap Acceptance Scenarios](https://mlsanity.com/paper/2211.05455/) (2022)
- [Using Models Based on Cognitive Theory to Predict Human Behavior in Traffic: A Case Study](https://mlsanity.com/paper/2305.15187/) (2023)
- [TrajFlow: Learning Distributions over Trajectories for Human Behavior Prediction](https://mlsanity.com/paper/2304.05166/) (2023)

## Built on by (most cited first, 2 of 2)

- [Evaluating randomized smoothing as a defense against adversarial attacks in trajectory prediction](https://mlsanity.com/paper/2603.10821/) (2026)
- [Clustered Randomized Smoothing for Stochastic Prediction Functions](https://mlsanity.com/paper/2608.12037/) (2026)

## Abstract (arXiv)

While trajectory prediction plays a critical role in enabling safe and effective path-planning in automated vehicles, standardized practices for evaluating such models remain underdeveloped. Recent efforts have aimed to unify dataset formats and model interfaces for easier comparisons, yet existing frameworks often fall short in supporting heterogeneous traffic scenarios, joint prediction models, or user documentation. In this work, we introduce STEP -- a new benchmarking framework that addresses these limitations by providing a unified interface for multiple datasets, enforcing consistent training and evaluation conditions, and supporting a wide range of prediction models. We demonstrate the capabilities of STEP in a number of experiments which reveal 1) the limitations of widely-used testing procedures, 2) the importance of joint modeling of agents for better predictions of interactions, and 3) the vulnerability of current state-of-the-art models against both distribution shifts and targeted attacks by adversarial agents. With STEP, we aim to shift the focus from the ``leaderboard'' approach to deeper insights about model behavior and generalization in complex multi-agent settings.