mlsanityData to 14 Oct 2026

Unveiling Transferability in Trajectory Prediction via Latent Scene Embeddings

arXiv 2606.30777 ↗First posted 29 Jun 2026cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

The growing availability of trajectory datasets has fueled major advances in data-driven motion prediction. Yet, models trained on one dataset often fail to generalize beyond their training domain as a result of differences in scene layouts, agent behaviors, and sensing conditions. A framework that learns latent representations of datasets and quantifies their similarity using distributional metrics is presented. This large-scale study covers 24 major datasets, including the most widely used motion-prediction benchmarks, and shows that the resulting transferability scores strongly correlate with cross-dataset model performance. The results provide practical guidance for dataset selection, pretraining, and large-scale foundation models for motion prediction, paving the way toward more generalizable and robust predictive systems.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 84 of its 116 references that are in the map. First posted on arXiv 29 Jun 2026.

Builds on

84 of 116 references.

References · 84

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  3. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  4. 2016Layer Normalizationcited by 5,669
  5. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 5,045
  6. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
  7. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  8. 2016Gaussian Error Linear Units (GELUs)cited by 3,502
  9. 2019Fast Graph Representation Learning with PyTorch Geometriccited by 2,434
  10. 2020Knowledge Distillation: A Surveycited by 1,603
  11. 2014On the Properties of Neural Machine Translation: Encoder-Decoder Approachescited by 1,304
  12. 2014How transferable are features in deep neural networks?cited by 1,225
38 more
  1. 2024The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scalecited by 1,160
  2. 2018Weisfeiler and Leman Go Neural: Higher-order Graph Neural Networkscited by 881
  3. 2019Argoverse: 3D Tracking and Forecasting with Rich Mapscited by 732
  4. 2013An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networkscited by 722
  5. 2021Deduplicating Training Data Makes Language Models Bettercited by 714
  6. 2023Argoverse 2: Next Generation Datasets for Self-Driving Perception and Forecastingcited by 707
  7. 2018Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networkscited by 690
  8. 2019DropEdge: Towards Deep Graph Convolutional Networks on Node Classificationcited by 680
  9. 2021Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Datasetcited by 537
  10. 2020Learning Lane Graph Representations for Motion Forecastingcited by 382
  11. 2021AgentFormer: Agent-Aware Transformers for Socio-Temporal Multi-Agent Forecastingcited by 331
  12. 2018SoPhie: An Attentive GAN for Predicting Paths Compliant to Social and Physical Constraintscited by 298
  13. 2018Multimodal Trajectory Predictions for Autonomous Driving using Deep Convolutional Networkscited by 289
  14. 2018The highD Dataset: A Drone Dataset of Naturalistic Vehicle Trajectories on German Highways for Validation of Highly Automated Driving Systemscited by 277
  15. 2020One Thousand and One Hours: Self-driving Motion Prediction Datasetcited by 240
  16. 2019INTERACTION Dataset: An INTERnational, Adversarial and Cooperative moTION Dataset in Interactive Driving Scenarios with Semantic Mapscited by 231
  17. 2018Characterizing and Avoiding Negative Transfercited by 229
  18. 2018Convolutional Social Pooling for Vehicle Trajectory Predictioncited by 221
  19. 2023MotionDiffuser: Controllable Multi-Agent Motion Prediction using Diffusioncited by 213
  20. 2021Multimodal Motion Prediction with Stacked Transformerscited by 208
  21. 2019Task2Vec: Task Embedding for Meta-Learningcited by 205
  22. 2022Stochastic Trajectory Prediction via Motion Indeterminacy Diffusioncited by 185
  23. 2020Transformer Networks for Trajectory Forecastingcited by 181
  24. 2021GOHOME: Graph-Oriented Heatmap Output for future Motion Estimationcited by 146
  25. 2019The inD Dataset: A Drone Dataset of Naturalistic Road User Trajectories at German Intersectionscited by 137
  26. 2023Leapfrog Diffusion Model for Stochastic Trajectory Predictioncited by 132
  27. 2019Multi-Agent Tensor Fusion for Contextual Trajectory Predictioncited by 131
  28. 2023Vision Language Models in Autonomous Driving: A Survey and Outlookcited by 126
  29. 2021Multimodal Trajectory Prediction Conditioned on Lane-Graph Traversalscited by 113
  30. 2018TrafficPredict: Trajectory Prediction for Heterogeneous Traffic-Agentscited by 111
  31. 2022TrafficGen: Learning to Generate Diverse and Realistic Traffic Scenarioscited by 106
  32. 2019What the Constant Velocity Model Can Teach Us About Pedestrian Motion Predictioncited by 80
  33. 2019Social Ways: Learning Multi-Modal Distributions of Pedestrian Trajectories with GANscited by 79
  34. 2021Multi-modal Motion Prediction with Transformer-based Neural Network for Autonomous Drivingcited by 73
  35. 2020MANTRA: Memory Augmented Networks for Multiple Trajectory Predictioncited by 69
  36. 2020Diverse and Admissible Trajectory Forecasting through Multimodal Context Understandingcited by 68
  37. 2022SocialVAE: Human Trajectory Prediction using Timewise Latentscited by 67
  38. 2023ScenarioNet: Open-Source Platform for Large-Scale Traffic Scenario Simulation and Modelingcited by 60

50 most cited of 84 All 84 (JSON)

Built on by

Not cited yet.