mlsanityData to 7 Oct 2026

THOMAS: Trajectory Heatmap Output with learned Multi-Agent Sampling

arXiv 2110.06607 ↗First posted 13 Oct 2021cs.CV cs.ROcited by 101

Not on the map. Why?

Abstract (arXiv)

In this paper, we propose THOMAS, a joint multi-agent trajectory prediction framework allowing for an efficient and consistent prediction of multi-agent multi-modal trajectories. We present a unified model architecture for simultaneous agent future heatmap estimation, in which we leverage hierarchical and sparse image generation for fast and memory-efficient inference. We propose a learnable trajectory recombination model that takes as input a set of predicted trajectories for each agent and outputs its consistent reordered recombination. This recombination module is able to realign the initially independent modalities so that they do no collide and are coherent with each other. We report our results on the Interaction multi-agent prediction challenge and rank 1st on the online test leaderboard.

Key facts. Cited by 101 papers in the map (data through 7 Oct 2026). Builds on 41 of its 49 references that are in the map. First posted on arXiv 13 Oct 2021.

Builds on

41 of 49 references.

References · 41

  1. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  2. 2019Argoverse: 3D Tracking and Forecasting with Rich Mapscited by 728
  3. 2018Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networkscited by 687
  4. 2021Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Datasetcited by 532
  5. 2020VectorNet: Encoding HD Maps and Agent Dynamics from Vectorized Representationcited by 528
  6. 2019MultiPath: Multiple Probabilistic Anchor Trajectory Hypotheses for Behavior Predictioncited by 417
  7. 2020Learning Lane Graph Representations for Motion Forecastingcited by 380
  8. 2020TNT: Target-driveN Trajectory Predictioncited by 345
  9. 2021AgentFormer: Agent-Aware Transformers for Socio-Temporal Multi-Agent Forecastingcited by 327
  10. 2017DESIRE: Distant Future Prediction in Dynamic Scenes with Interacting Agentscited by 310
  11. 2018SoPhie: An Attentive GAN for Predicting Paths Compliant to Social and Physical Constraintscited by 296
  12. 2021DenseTNT: End-to-end Trajectory Prediction from Dense Goal Setscited by 292
29 more
  1. 2018Multimodal Trajectory Predictions for Autonomous Driving using Deep Convolutional Networkscited by 289
  2. 2019CoverNet: Multimodal Behavior Prediction using Trajectory Setscited by 259
  3. 2019INTERACTION Dataset: An INTERnational, Adversarial and Cooperative moTION Dataset in Interactive Driving Scenarios with Semantic Mapscited by 231
  4. 2018Convolutional Social Pooling for Vehicle Trajectory Predictioncited by 221
  5. 2019PRECOG: PREdiction Conditioned On Goals in Visual Multi-Agent Settingscited by 178
  6. 2019Multiple Futures Predictioncited by 178
  7. 2021GOHOME: Graph-Oriented Heatmap Output for future Motion Estimationcited by 146
  8. 2021HOME: Heatmap Output for future Motion Estimationcited by 122
  9. 2019Rules of the Road: Predicting Driving Behavior with a Convolutional Model of Semantic Interactionscited by 117
  10. 2020Implicit Latent Variable Model for Scene-Consistent Motion Forecastingcited by 116
  11. 2021LaneRCNN: Distributed Representations for Graph-Centric Motion Forecastingcited by 112
  12. 2021Multimodal Trajectory Prediction Conditioned on Lane-Graph Traversalscited by 112
  13. 2021LookOut: Diverse Multi-Future Prediction and Planning for Self-Drivingcited by 96
  14. 2021TPCN: Temporal Point Cloud Networks for Motion Forecastingcited by 94
  15. 2019Multi-Head Attention for Multi-Modal Joint Vehicle Motion Forecastingcited by 87
  16. 2018An LSTM Network for Highway Trajectory Predictioncited by 81
  17. 2020What-If Motion Prediction for Autonomous Drivingcited by 80
  18. 2020Trajectory Forecasts in Unknown Environments Conditioned on Grid-Based Planscited by 72
  19. 2021LaPred: Lane-Aware Prediction of Multi-Modal Future Trajectories of Dynamic Agentscited by 64
  20. 2021Learning to Predict Vehicle Trajectories with Model-based Planningcited by 63
  21. 2021Stepwise Goal-Driven Networks for Trajectory Predictioncited by 62
  22. 2020End-to-end Contextual Perception and Prediction with Interaction Transformercited by 54
  23. 2021Divide-and-Conquer for Lane-Aware Diverse Trajectory Predictioncited by 37
  24. 2020MATS: An Interpretable Trajectory Forecasting Representation for Planning and Controlcited by 30
  25. 2020Map-Adaptive Goal-Based Trajectory Predictioncited by 29
  26. 2020Probabilistic Multi-modal Trajectory Prediction with Lane Attention for Autonomous Vehiclescited by 28
  27. 2020ReCoG: A Deep Learning Framework with Heterogeneous Graph for Interaction-Aware Trajectory Predictioncited by 20
  28. 2020Trajectory Prediction for Autonomous Driving based on Multi-Head Attention with Joint Agent-Map Representationcited by 12
  29. 2021Asymmetrical Bi-RNN for pedestrian trajectory encodingcited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 19; 2023: 26; 2024: 28; 2025: 23; 2026: 5.

101 papers.

Sort:

By area:

  1. 2023Argoverse 2: Next Generation Datasets for Self-Driving Perception and Forecastingcited by 701
  2. 2022Motion Transformer with Global Intention Localization and Local Movement Refinementcited by 267
  3. 2023GameFormer: Game-theoretic Modeling and Learning of Transformer-based Interactive Prediction and Planning for Autonomous Drivingcited by 141
  4. 2023MTR++: Multi-Agent Motion Prediction with Symmetric Scene Modeling and Guided Intention Queryingcited by 139
  5. 2022ViP3D: End-to-end Visual Trajectory Prediction via 3D Agent Queriescited by 116
  6. 2022HDGT: Heterogeneous Driving Graph Transformer for Multi-Agent Trajectory Prediction via Scene Encodingcited by 109
  7. 2022BITS: Bi-level Imitation for Traffic Simulationcited by 80
  8. 2023Forecast-MAE: Self-supervised Pre-training for Motion Forecasting with Masked Autoencoderscited by 72
  9. 2022Differentiable Integrated Motion Prediction and Planning with Learnable Cost Function for Autonomous Drivingcited by 71
  10. 2022GoRela: Go Relative for Viewpoint-Invariant Motion Forecastingcited by 68
  11. 2023LAformer: Trajectory Prediction for Autonomous Driving with Lane-Aware Scene Constraintscited by 50
  12. 2022GANet: Goal Area Network for Motion Forecastingcited by 49
38 more
  1. 2023Leveraging Future Relationship Reasoning for Vehicle Trajectory Predictioncited by 47
  2. 2023Scenario Diffusion: Controllable Driving Scenario Generation With Diffusioncited by 46
  3. 2023Traj-MAE: Masked Autoencoders for Trajectory Predictioncited by 42
  4. 2022FJMP: Factorized Joint Multi-Agent Motion Prediction over Learned Directed Acyclic Interaction Graphscited by 41
  5. 2023ADAPT: Efficient Multi-Agent Trajectory Prediction with Adaptationcited by 39
  6. 2024HPNet: Dynamic Trajectory Forecasting with Historical Prediction Attentioncited by 37
  7. 2023MacFormer: Map-Agent Coupled Transformer for Real-time and Robust Trajectory Predictioncited by 32
  8. 2023Real-Time Motion Prediction via Heterogeneous Polyline Transformer with Relative Pose Encodingcited by 32
  9. 2024Producing and Leveraging Online Map Uncertainty in Trajectory Predictioncited by 32
  10. 2023The Integration of Prediction and Planning in Deep Learning Automated Driving Systems: A Reviewcited by 29
  11. 2023Dynamic Scenario Representation Learning for Motion Forecasting with Heterogeneous Graph Convolutional Recurrent Networkscited by 27
  12. 2022PreTraM: Self-Supervised Pre-training via Connecting Trajectory and Mapcited by 25
  13. 2022R-Pred: Two-Stage Motion Prediction Via Tube-Query Attention-Based Trajectory Refinementcited by 23
  14. 2024T4P: Test-Time Training of Trajectory Prediction via Masked Autoencoder and Actor-specific Token Memorycited by 20
  15. 2024Hybrid-Prediction Integrated Planning for Autonomous Drivingcited by 18
  16. 2024VisionTrap: Vision-Augmented Trajectory Prediction Guided by Textual Descriptionscited by 18
  17. 2024Optimizing Diffusion Models for Joint Trajectory Prediction and Controllable Generationcited by 18
  18. 2025Trajectory Mamba: Efficient Attention-Mamba Forecasting Model Based on Selective SSMcited by 17
  19. 2024DeTra: A Unified Model for Object Detection and Trajectory Forecastingcited by 15
  20. 2024doScenes: An Autonomous Driving Dataset with Natural Language Instruction for Human Interaction and Vision-Language Navigationcited by 14
  21. 2025Trajectory Prediction for Autonomous Driving: Progress, Limitations, and Future Directionscited by 14
  22. 2025Data Scaling Laws for End-to-End Autonomous Drivingcited by 14
  23. 2022Uncertainty estimation for Cross-dataset performance in Trajectory predictioncited by 13
  24. 2024AMP: Autoregressive Motion Prediction Revisited with Next Token Prediction for Autonomous Drivingcited by 13
  25. 2022RAP: Risk-Aware Prediction for Robust Planningcited by 12
  26. 2023Physics Constrained Motion Prediction with Uncertainty Quantificationcited by 11
  27. 2024Motion Forecasting in Continuous Drivingcited by 11
  28. 2022Importance is in your attention: agent importance prediction for autonomous drivingcited by 10
  29. 2022Towards Explainable Motion Prediction using Heterogeneous Graph Representationscited by 10
  30. 2024SemanticFormer: Holistic and Semantic Traffic Scene Representation for Trajectory Prediction using Knowledge Graphscited by 10
  31. 2024Efficient Motion Prediction: A Lightweight & Accurate Trajectory Prediction Model With Fast Training and Inference Speedcited by 9
  32. 2022Aware of the History: Trajectory Forecasting with the Local Behavior Datacited by 8
  33. 2023Occupancy Prediction-Guided Neural Planner for Autonomous Drivingcited by 8
  34. 2023BiFF: Bi-level Future Fusion with Polyline-based Coordinate for Interactive Trajectory Predictioncited by 8
  35. 2024Accelerating Online Mapping and Behavior Prediction via Direct BEV Feature Attentioncited by 8
  36. 2025Motion Forecasting for Autonomous Vehicles: A Surveycited by 8
  37. 2025Diverse Controllable Diffusion Policy with Signal Temporal Logiccited by 8
  38. 2023Conditional Unscented Autoencoders for Trajectory Predictioncited by 7

50 most cited of 101 All 101 (JSON)