mlsanityData to 14 Oct 2026

trajdata: A Unified Interface to Multiple Human Trajectory Datasets

arXiv 2307.13924 ↗First posted 26 Jul 2023cs.CV cs.LG cs.ROcited by 25

Not on the map. Why?

Abstract (arXiv)

The field of trajectory forecasting has grown significantly in recent years, partially owing to the release of numerous large-scale, real-world human trajectory datasets for autonomous vehicles (AVs) and pedestrian motion tracking. While such datasets have been a boon for the community, they each use custom and unique data formats and APIs, making it cumbersome for researchers to train and evaluate methods across multiple datasets. To remedy this, we present trajdata: a unified interface to multiple human trajectory datasets. At its core, trajdata provides a simple, uniform, and efficient representation and API for trajectory and map data. As a demonstration of its capabilities, in this work we conduct a comprehensive empirical evaluation of existing trajectory datasets, providing users with a rich understanding of the data underpinning much of current pedestrian and AV motion forecasting research, and proposing suggestions for future datasets from these insights. trajdata is permissively licensed (Apache 2.0) and can be accessed online at https://github.com/NVlabs/trajdata

Key facts. Cited by 25 papers in the map (data through 14 Oct 2026). Builds on 19 of its 43 references that are in the map. First posted on arXiv 26 Jul 2023.

Builds on

19 of 43 references.

References · 19

  1. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
  2. 2016OpenAI Gymcited by 1,736
  3. 2021Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Datasetcited by 537
  4. 2021NuPlan: A closed-loop ML-based planning benchmark for autonomous vehiclescited by 493
  5. 2020One Thousand and One Hours: Self-driving Motion Prediction Datasetcited by 240
  6. 2019Human Motion Trajectory Prediction: A Surveycited by 232
  7. 2019INTERACTION Dataset: An INTERnational, Adversarial and Cooperative moTION Dataset in Interactive Driving Scenarios with Semantic Mapscited by 231
  8. 2022Guided Conditional Diffusion for Controllable Traffic Simulationcited by 190
  9. 2023Trace and Pace: Controllable Pedestrian Animation via Guided Trajectory Diffusioncited by 138
  10. 2020Human Trajectory Forecasting in Crowds: A Deep Learning Perspectivecited by 129
  11. 2021Shifts: A Dataset of Real Distributional Shift Across Multiple Large-Scale Taskscited by 101
  12. 2022BITS: Bi-level Imitation for Traffic Simulationcited by 80
7 more
  1. 2021Identifying Driver Interactions via Conditional Behavior Predictioncited by 48
  2. 2023Tree-structured Policy Planning with Learned Behavior Modelscited by 24
  3. 2022Expanding the Deployment Envelope of Behavior Prediction via Adaptive Meta-Learningcited by 23
  4. 2020OpenTraj: Assessing Prediction Complexity in Human Trajectories Datasetscited by 21
  5. 2022Uncertainty estimation for Cross-dataset performance in Trajectory predictioncited by 13
  6. 2022The Atlas Benchmark: an Automated Evaluation Framework for Human Motion Predictioncited by 7
  7. 2022Planning with Occluded Traffic Agents using Bi-Level Variational Occlusion Modelscited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 2; 2024: 8; 2025: 5; 2026: 10.

25 papers.

Sort:

By area:

  1. 2024UniTraj: A Unified Framework for Scalable Vehicle Trajectory Predictioncited by 52
  2. 2023RealGen: Retrieval Augmented Generation for Controllable Traffic Scenarioscited by 46
  3. 2023SAFE-SIM: Safety-Critical Closed-Loop Traffic Simulation with Diffusion-Controllable Adversariescited by 38
  4. 2024Producing and Leveraging Online Map Uncertainty in Trajectory Predictioncited by 33
  5. 2024Multi-Transmotion: Pre-trained Model for Human Motion Predictioncited by 21
  6. 2024T4P: Test-Time Training of Trajectory Prediction via Masked Autoencoder and Actor-specific Token Memorycited by 20
  7. 2025Data Scaling Laws for End-to-End Autonomous Drivingcited by 15
  8. 2025LANGTRAJ: Diffusion Model and Dataset for Language-Conditioned Trajectory Simulationcited by 11
  9. 2024Accelerating Online Mapping and Behavior Prediction via Direct BEV Feature Attentioncited by 8
  10. 2024Prediction Horizon Requirements for Automated Driving: Optimizing Safety, Comfort, and Efficiencycited by 3
  11. 2025OmniTraj: Pre-Training on Heterogeneous Data for Adaptive and Zero-Shot Human Trajectory Predictioncited by 3
  12. 2026Implicit Maximum Likelihood Estimation for Real-time Generative Model Predictive Controlcited by 3
13 more
  1. 2026123D: Unifying Multi-Modal Autonomous Driving Data at Scalecited by 3
  2. 2024Toward Unified Practices in Trajectory Prediction Research on Bird's-Eye-View Datasetscited by 2
  3. 2025STEP: Structured Training and Evaluation Platform for benchmarking trajectory prediction modelscited by 2
  4. 2026DeepUrban: Interaction-Aware Trajectory Prediction and Planning for Automated Driving by Aerial Imagerycited by 2
  5. 2025Poly-Autoregressive Prediction for Modeling Interactionscited by 1
  6. 2026HetroD: A High-Fidelity Drone Dataset and Benchmark for Autonomous Driving in Heterogeneous Trafficcited by 1
  7. 2024Amelia: A Large Dataset and Benchmark for Airport Surface Movement Forecastingcited by 0
  8. 2026MetaDAT: Generalizable Trajectory Prediction via Meta Pre-training and Data-Adaptive Test-Time Updatingcited by 0
  9. 2026Uncertainty-Aware Trajectory Prediction: A Unified Framework Harnessing Positional and Semantic Uncertaintiescited by 0
  10. 2026StandardE2E: A Unified Framework for End-to-End Autonomous Driving Datasetscited by 0
  11. 2026Unveiling Transferability in Trajectory Prediction via Latent Scene Embeddingscited by 0
  12. 2026End-to-end Conditional Diffusion for Realistic and Controllable Visual Traffic Scenario Generationcited by 0
  13. 2026STRIDE: Automated Evaluation of Text-to-Trajectory Alignment across Diverse Contextscited by 0