mlsanityData to 7 Oct 2026

On Exposing the Challenging Long Tail in Future Prediction of Traffic Actors

arXiv 2103.12474 ↗First posted 23 Mar 2021cs.CVcited by 48

Not on the map. Why?

Abstract (arXiv)

Predicting the states of dynamic traffic actors into the future is important for autonomous systems to operate safelyand efficiently. Remarkably, the most critical scenarios aremuch less frequent and more complex than the uncriticalones. Therefore, uncritical cases dominate the prediction. In this paper, we address specifically the challenging scenarios at the long tail of the dataset distribution. Our analysis shows that the common losses tend to place challenging cases suboptimally in the embedding space. As a consequence, we propose to supplement the usual loss with aloss that places challenging cases closer to each other. This triggers sharing information among challenging cases andlearning specific predictive features. We show on four public datasets that this leads to improved performance on the challenging scenarios while the overall performance stays stable. The approach is agnostic w.r.t. the used network architecture, input modality or viewpoint, and can be integrated into existing solutions easily. Code is available at https://github.com/lmb-freiburg/Contrastive-Future-Trajectory-Prediction

Key facts. Cited by 48 papers in the map (data through 7 Oct 2026). Builds on 59 of its 80 references that are in the map. First posted on arXiv 23 Mar 2021.

Builds on

59 of 80 references.

References · 59

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  3. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  4. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  5. 2020Supervised Contrastive Learningcited by 2,803
  6. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,436
  7. 2019Class-Balanced Loss Based on Effective Number of Samplescited by 1,239
  8. 2019Learning Imbalanced Datasets with Label-Distribution-Aware Margin Losscited by 987
  9. 2019Decoupling Representation and Classifier for Long-Tailed Recognitioncited by 806
  10. 2017Focal Loss for Dense Object Detectioncited by 749
  11. 2018Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networkscited by 687
  12. 2016Training Region-based Object Detectors with Online Hard Example Miningcited by 577
38 more
  1. 2020Learning Lane Graph Representations for Motion Forecastingcited by 380
  2. 2020Social-STGCNN: A Social Spatio-Temporal Graph Convolutional Neural Network for Human Trajectory Predictioncited by 324
  3. 2017DESIRE: Distant Future Prediction in Dynamic Scenes with Interacting Agentscited by 310
  4. 2018SoPhie: An Attentive GAN for Predicting Paths Compliant to Social and Physical Constraintscited by 296
  5. 2020It Is Not the Journey but the Destination: Endpoint Conditioned Trajectory Predictioncited by 235
  6. 2019Social-BiGAT: Multimodal Trajectory Forecasting using Bicycle-GAN and Graph Attention Networkscited by 228
  7. 2020Spatio-Temporal Graph Transformer Networks for Pedestrian Trajectory Predictioncited by 218
  8. 2020Rethinking the Value of Labels for Improving Class-Imbalanced Learningcited by 216
  9. 2019PRECOG: PREdiction Conditioned On Goals in Visual Multi-Agent Settingscited by 178
  10. 2018Large Scale Fine-Grained Categorization and Domain-Specific Transfer Learningcited by 174
  11. 2015Cost Sensitive Learning of Deep Feature Representations from Imbalanced Datacited by 167
  12. 2020Overcoming Classifier Imbalance for Long-tail Object Detection with Balanced Group Softmaxcited by 152
  13. 2020Rethinking Class-Balanced Methods for Long-Tailed Visual Recognition from a Domain Adaptation Perspectivecited by 143
  14. 2020M2m: Imbalanced Classification via Major-to-minor Translationcited by 134
  15. 2019Multi-Agent Tensor Fusion for Contextual Trajectory Predictioncited by 131
  16. 2016Range Loss for Deep Face Recognition with Long-tailcited by 111
  17. 2018TrafficPredict: Trajectory Prediction for Heterogeneous Traffic-Agentscited by 111
  18. 2019SR-LSTM: State Refinement for LSTM towards Pedestrian Trajectory Predictioncited by 111
  19. 2019Spatially-Aware Graph Neural Networks for Relational Behavior Forecasting from Sensor Datacited by 104
  20. 2019Overcoming Limitations of Mixture Density Networks: A Sampling and Fitting Framework for Multimodal Future Predictioncited by 96
  21. 2018Uncertainty-aware Short-term Motion Prediction of Traffic Actors for Autonomous Drivingcited by 90
  22. 2018TraPHic: Trajectory Prediction in Dense and Heterogeneous Traffic Using Weighted Interactionscited by 82
  23. 2020TPNet: Trajectory Proposal Network for Motion Predictioncited by 80
  24. 2019Striking the Right Balance with Uncertaintycited by 79
  25. 2018Multi-Modal Trajectory Prediction of Surrounding Vehicles with Maneuver based LSTMscited by 77
  26. 2019Generative Adversarial Minority Oversamplingcited by 72
  27. 2019The Garden of Forking Paths: Towards Multi-Future Trajectory Predictioncited by 72
  28. 2020Recursive Social Behavior Graph for Trajectory Predictioncited by 66
  29. 2018Imbalanced Deep Learning by Minority Class Incremental Rectificationcited by 65
  30. 2017Long-Term On-Board Prediction of People in Traffic Scenes under Uncertaintycited by 63
  31. 2020TITAN: Future Forecast using Action Priorscited by 58
  32. 2018How would surround vehicles move? A Unified Framework for Maneuver Classification and Motion Predictioncited by 46
  33. 2018Egocentric Vision-based Future Vehicle Localization for Intelligent Driving Assistance Systemscited by 42
  34. 2020Large-Scale Object Detection in the Wild from Imbalanced Multi-Labelscited by 35
  35. 2017CAR-Net: Clairvoyant Attentive Recurrent Networkcited by 34
  36. 2017Context-Aware Trajectory Predictioncited by 31
  37. 2020Reciprocal Learning Networks for Human Trajectory Predictioncited by 30
  38. 2018Scene-LSTM: A Model for Human Trajectory Predictioncited by 24

50 most cited of 59 All 59 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 2; 2022: 7; 2023: 12; 2024: 10; 2025: 10; 2026: 7.

48 papers.

Sort:

By area:

  1. 2023Street-View Image Generation from a Bird's-Eye View Layoutcited by 102
  2. 2023The Safety Filter: A Unified View of Safety-Critical Control in Autonomous Systemscited by 77
  3. 2024CoVLA: Comprehensive Vision-Language-Action Dataset for Autonomous Drivingcited by 74
  4. 2022Adaptive Trajectory Prediction via Transferable GNNcited by 51
  5. 2024UniTraj: A Unified Framework for Scalable Vehicle Trajectory Predictioncited by 51
  6. 2023Synthetic Datasets for Autonomous Driving: A Surveycited by 48
  7. 2023Uncovering the Missing Pattern: Unified Framework Towards Trajectory Imputation and Predictioncited by 47
  8. 2022Social-Implicit: Rethinking Trajectory Prediction Evaluation and The Effectiveness of Implicit Maximum Likelihood Estimationcited by 40
  9. 2023FEND: A Future Enhanced Distribution-Aware Contrastive Learning Framework for Long-tail Trajectory Predictioncited by 31
  10. 2021You Mostly Walk Alone: Analyzing Feature Attribution in Trajectory Predictioncited by 23
  11. 2022Long-Tail Prediction Uncertainty Aware Trajectory Planning for Self-driving Vehiclescited by 22
  12. 2022CausalAgents: A Robustness Benchmark for Motion Forecasting using Causal Relationshipscited by 18
36 more
  1. 2023Fed-GraB: Federated Long-tailed Learning with Self-Adjusting Gradient Balancercited by 17
  2. 2023Safety-Critical Scenario Generation Via Reinforcement Learning Based Editingcited by 16
  3. 2022Action-based Contrastive Learning for Trajectory Predictioncited by 15
  4. 2023Holistic Representation Learning for Multitask Trajectory Anomaly Detectioncited by 15
  5. 2024Generating Out-Of-Distribution Scenarios Using Language Modelscited by 10
  6. 2023Visual Data-Type Understanding does not emerge from Scaling Vision-Language Modelscited by 9
  7. 2023Vision-based Multi-future Trajectory Prediction: A Surveycited by 8
  8. 2022Taming the Long Tail of Deep Probabilistic Forecastingcited by 7
  9. 2024TrACT: A Training Dynamics Aware Contrastive Learning Framework for Long-tail Trajectory Predictioncited by 7
  10. 2021Raising context awareness in motion forecastingcited by 6
  11. 2024Multi-Agent Trajectory Prediction with Difficulty-Guided Feature Enhancement Networkcited by 6
  12. 2025Generative Active Learning for Long-tail Trajectory Prediction via Controllable Diffusion Modelcited by 6
  13. 2023CC-SGG: Corner Case Scenario Generation using Learned Scene Graphscited by 4
  14. 2024AMEND: A Mixture of Experts Framework for Long-tailed Trajectory Predictioncited by 4
  15. 2025AMD: Adaptive Momentum and Decoupled Contrastive Learning Framework for Robust Long-Tail Trajectory Predictioncited by 4
  16. 2022Joint Forecasting of Panoptic Segmentations with Difference Attentioncited by 3
  17. 2024G2LTraj: A Global-to-Local Generation Approach for Trajectory Predictioncited by 3
  18. 2024Den-TP: A Density-Balanced Data Curation and Evaluation Framework for Trajectory Predictioncited by 3
  19. 2025V-Max: A Reinforcement Learning Framework for Autonomous Drivingcited by 3
  20. 2026A Sustainable AI Economy Needs Data Deals That Work for Generatorscited by 3
  21. 2024Critical Example Mining for Vehicle Trajectory Prediction using Flow-based Generative Modelscited by 1
  22. 2025Risk-Informed Diffusion Transformer for Long-Tail Trajectory Prediction in the Crash Scenariocited by 1
  23. 2025AuthSim: Towards Authentic and Effective Safety-critical Scenario Generation for Autonomous Driving Testscited by 1
  24. 2025PAVE: An End-to-End Dataset for Production Autonomous Vehicle Evaluationcited by 1
  25. 2025MTR-VP: Towards End-to-End Trajectory Planning through Context-Driven Image Encoding and Multiple Trajectory Predictioncited by 1
  26. 2026EVEREST: An Evidential, Tail-Aware Transformer for Rare-Event Time-Series Forecastingcited by 1
  27. 2026CLAP: Contrastive Latent-space Prompt Optimization for End-to-end Autonomous Drivingcited by 1
  28. 2023FollowMe: Vehicle Behaviour Prediction in Autonomous Vehicle Settingscited by 0
  29. 2024Designing deep neural networks for driver intention recognitioncited by 0
  30. 2025Ensemble of Pre-Trained Models for Long-Tailed Trajectory Predictioncited by 0
  31. 2025AdaSports-Traj: Role- and Domain-Aware Adaptation for Multi-Agent Trajectory Modeling in Sportscited by 0
  32. 2025A Trajectory Generator for High-Density Traffic and Diverse Agent-Interaction Scenarioscited by 0
  33. 2026MetaDAT: Generalizable Trajectory Prediction via Meta Pre-training and Data-Adaptive Test-Time Updatingcited by 0
  34. 2026SAIL: Scene-aware Adaptive Iterative Learning for Long-Tail Trajectory Prediction in Autonomous Vehiclescited by 0
  35. 2026What Probing Reveals about Autonomous Driving: Linking Internal Prediction Errors to Ego Planningcited by 0
  36. 2026VOLA: Improving Open-World Driving by VLM-Based Semantic Attribute Predictioncited by 0