mlsanityData to 14 Oct 2026

Under the Hood of Transformer Networks for Trajectory Forecasting

arXiv 2203.11878 ↗First posted 22 Mar 2022cs.CVcited by 8

Not on the map. Why?

Abstract (arXiv)

Transformer Networks have established themselves as the de-facto state-of-the-art for trajectory forecasting but there is currently no systematic study on their capability to model the motion patterns of people, without interactions with other individuals nor the social context. This paper proposes the first in-depth study of Transformer Networks (TF) and Bidirectional Transformers (BERT) for the forecasting of the individual motion of people, without bells and whistles. We conduct an exhaustive evaluation of input/output representations, problem formulations and sequence modeling, including a novel analysis of their capability to predict multi-modal futures. Out of comparative evaluation on the ETH+UCY benchmark, both TF and BERT are top performers in predicting individual motions, definitely overcoming RNNs and LSTMs. Furthermore, they remain within a narrow margin wrt more complex techniques, which include both social interactions and scene contexts. Source code will be released for all conducted experiments.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 36 of its 56 references that are in the map. First posted on arXiv 22 Mar 2022.

Builds on

36 of 56 references.

References · 36

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  4. 2018Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networkscited by 690
  5. 2013Generating Sequences With Recurrent Neural Networkscited by 554
  6. 2019MultiPath: Multiple Probabilistic Anchor Trajectory Hypotheses for Behavior Predictioncited by 420
  7. 2021A Survey of Transformerscited by 401
  8. 2021AgentFormer: Agent-Aware Transformers for Socio-Temporal Multi-Agent Forecastingcited by 331
  9. 2021DenseTNT: End-to-end Trajectory Prediction from Dense Goal Setscited by 295
  10. 2020It Is Not the Journey but the Destination: Endpoint Conditioned Trajectory Predictioncited by 236
  11. 2019Social-BiGAT: Multimodal Trajectory Forecasting using Bicycle-GAN and Graph Attention Networkscited by 229
  12. 2020Spatio-Temporal Graph Transformer Networks for Pedestrian Trajectory Predictioncited by 218
24 more
  1. 2020Long-term Human Motion Prediction with Scene Contextcited by 192
  2. 2017Social Attention: Modeling Attention in Human Crowdscited by 186
  3. 2020DLow: Diversifying Latent Flows for Diverse Human Motion Predictioncited by 185
  4. 2020Transformer Networks for Trajectory Forecastingcited by 181
  5. 2020From Goals, Waypoints & Paths To Long Term Human Trajectory Forecastingcited by 167
  6. 2020Human Trajectory Forecasting in Crowds: A Deep Learning Perspectivecited by 129
  7. 2019What the Constant Velocity Model Can Teach Us About Pedestrian Motion Predictioncited by 80
  8. 2019Diverse Trajectory Forecasting with Determinantal Point Processescited by 80
  9. 2017Tracking The Untrackable: Learning To Track Multiple Cues with Long-Term Dependenciescited by 76
  10. 2019The Garden of Forking Paths: Towards Multi-Future Trajectory Predictioncited by 72
  11. 2020Trajectron++: Dynamically-Feasible Trajectory Forecasting With Heterogeneous Datacited by 58
  12. 2020End-to-end Contextual Perception and Prediction with Interaction Transformercited by 54
  13. 2021Divide-and-Conquer for Lane-Aware Diverse Trajectory Predictioncited by 37
  14. 2019Disentangling Human Dynamics for Pedestrian Locomotion Forecasting with Noisy Supervisioncited by 27
  15. 2020OpenTraj: Assessing Prediction Complexity in Human Trajectories Datasetscited by 21
  16. 2020Pedestrian Trajectory Prediction with Convolutional Neural Networkscited by 21
  17. 2022CSCNet: Contextual Semantic Consistency Network for Trajectory Prediction in Crowded Spacescited by 18
  18. 2018MX-LSTM: mixing tracklets and vislets to jointly forecast trajectories and head posescited by 17
  19. 2018An Evaluation of Trajectory Prediction Approaches and Notes on the TrajNet Benchmarkcited by 16
  20. 2019Forecasting People Trajectories and Head Poses by Jointly Reasoning on Tracklets and Visletscited by 13
  21. 2020Driving Behavior Explanation with Multi-level Fusioncited by 12
  22. 2020Haar Wavelet based Block Autoregressive Flows for Trajectoriescited by 6
  23. 2021Pedestrian Trajectory Prediction via Spatial Interaction Transformer Networkcited by 4
  24. 2014Fuzzy human motion analysis: A reviewcited by 1

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 1; 2024: 3; 2025: 3.

8 papers.

  1. 2024Certified Human Trajectory Predictioncited by 6
  2. 2023Distilling Knowledge for Short-to-Long Term Trajectory Predictioncited by 5
  3. 2022Toward Smart Doors: A Position Papercited by 2
  4. 2024A Data-driven Crowd Simulation Framework Integrating Physics-informed Machine Learning with Navigation Potential Fieldscited by 2
  5. 2025ECAM: A Contrastive Learning Approach to Avoid Environmental Collision in Trajectory Forecastingcited by 2
  6. 2025TrajGATFormer: A Graph-Based Transformer Approach for Worker and Obstacle Trajectory Prediction in Off-site Construction Environmentscited by 2
  7. 2024Pedestrian Motion Prediction Using Transformer-based Behavior Clustering and Data-Driven Reachability Analysiscited by 1
  8. 2025Patient-specific vs Multi-Patient Vision Transformer for Markerless Tumor Motion Forecastingcited by 0