mlsanityData to 7 Oct 2026

Hierarchical Light Transformer Ensembles for Multimodal Trajectory Forecasting

arXiv 2403.17678 ↗First posted 26 Mar 2024cs.CVcited by 4

Not on the map. Why?

Abstract (arXiv)

Accurate trajectory forecasting is crucial for the performance of various systems, such as advanced driver-assistance systems and self-driving vehicles. These forecasts allow us to anticipate events that lead to collisions and, therefore, to mitigate them. Deep Neural Networks have excelled in motion forecasting, but overconfidence and weak uncertainty quantification persist. Deep Ensembles address these concerns, yet applying them to multimodal distributions remains challenging. In this paper, we propose a novel approach named Hierarchical Light Transformer Ensembles (HLT-Ens) aimed at efficiently training an ensemble of Transformer architectures using a novel hierarchical loss function. HLT-Ens leverages grouped fully connected layers, inspired by grouped convolution techniques, to capture multimodal distributions effectively. We demonstrate that HLT-Ens achieves state-of-the-art performance levels through extensive experimentation, offering a promising avenue for improving trajectory forecasting techniques.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 39 of its 52 references that are in the map. First posted on arXiv 26 Mar 2024.

Builds on

39 of 52 references.

References · 39

  1. 2017Attention Is All You Needcited by 59,932
  2. 2016Layer Normalizationcited by 5,579
  3. 2017On Calibration of Modern Neural Networkscited by 4,860
  4. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  5. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
  6. 2016Aggregated Residual Transformations for Deep Neural Networkscited by 2,540
  7. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,250
  8. 2018Group Normalizationcited by 1,514
  9. 2018Averaging Weights Leads to Wider Optima and Better Generalizationcited by 1,194
  10. 2016A Point Set Generation Network for 3D Object Reconstruction from a Single Imagecited by 972
  11. 2014Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Imagescited by 774
  12. 2019Argoverse: 3D Tracking and Forecasting with Rich Mapscited by 728
27 more
  1. 2023Argoverse 2: Next Generation Datasets for Self-Driving Perception and Forecastingcited by 701
  2. 2018Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networkscited by 687
  3. 2019MultiPath: Multiple Probabilistic Anchor Trajectory Hypotheses for Behavior Predictioncited by 417
  4. 2020BatchEnsemble: An Alternative Approach to Efficient Ensemble and Lifelong Learningcited by 368
  5. 2021AgentFormer: Agent-Aware Transformers for Socio-Temporal Multi-Agent Forecastingcited by 327
  6. 2017DESIRE: Distant Future Prediction in Dynamic Scenes with Interacting Agentscited by 310
  7. 2018Multimodal Trajectory Predictions for Autonomous Driving using Deep Convolutional Networkscited by 289
  8. 2022Wayformer: Motion Forecasting via Simple & Efficient Attention Networkscited by 279
  9. 2021MultiPath++: Efficient Information Fusion and Trajectory Aggregation for Behavior Predictioncited by 246
  10. 2020It Is Not the Journey but the Destination: Endpoint Conditioned Trajectory Predictioncited by 235
  11. 2019INTERACTION Dataset: An INTERnational, Adversarial and Cooperative moTION Dataset in Interactive Driving Scenarios with Semantic Mapscited by 231
  12. 2018Convolutional Social Pooling for Vehicle Trajectory Predictioncited by 221
  13. 2019Multiple Futures Predictioncited by 178
  14. 2020Training independent subnetworks for robust predictioncited by 177
  15. 2021Scene Transformer: A unified architecture for predicting multiple agent trajectoriescited by 165
  16. 2015Why M Heads are Better than One: Training a Diverse Ensemble of Deep Networkscited by 138
  17. 2021Latent Variable Sequential Set Transformers For Joint Multi-Agent Motion Predictioncited by 131
  18. 2021Shifts: A Dataset of Real Distributional Shift Across Multiple Large-Scale Taskscited by 100
  19. 2016Learning in an Uncertain World: Representing Ambiguity Through Multiple Hypothesescited by 96
  20. 2019Overcoming Limitations of Mixture Density Networks: A Sampling and Fitting Framework for Multimodal Future Predictioncited by 96
  21. 2019Multi-Head Attention for Multi-Modal Joint Vehicle Motion Forecastingcited by 87
  22. 2016Stochastic Multiple Choice Learning for Training Diverse Deep Ensemblescited by 77
  23. 2018Multi-Modal Trajectory Prediction of Surrounding Vehicles with Maneuver based LSTMscited by 77
  24. 2022Packed-Ensembles for Efficient Uncertainty Estimationcited by 49
  25. 2023ADAPT: Efficient Multi-Agent Trajectory Prediction with Adaptationcited by 39
  26. 2021Injecting Knowledge in Data-driven Vehicle Trajectory Predictorscited by 18
  27. 2022TENET: Transformer Encoding Network for Effective Temporal Flow on Motion Predictioncited by 14

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 3; 2026: 1.

4 papers.

  1. 2025Uncertainty Quantification for Machine Learning in Healthcare: A Surveycited by 11
  2. 2025Social-Pose: Enhancing Trajectory Prediction with Human Body Posecited by 5
  3. 2025Ensembling Pruned Attention Heads For Uncertainty-Aware Efficient Transformerscited by 5
  4. 2026EdgeVTP: Exploration of Latency-efficient Trajectory Prediction for Edge-based Embedded Vision Applicationscited by 0