mlsanityData to 7 Oct 2026

SceneMotion: From Agent-Centric Embeddings to Scene-Wide Forecasts

arXiv 2408.01537 ↗First posted 2 Aug 2024cs.CV cs.ROcited by 4

Not on the map. Why?

Abstract (arXiv)

Self-driving vehicles rely on multimodal motion forecasts to effectively interact with their environment and plan safe maneuvers. We introduce SceneMotion, an attention-based model for forecasting scene-wide motion modes of multiple traffic agents. Our model transforms local agent-centric embeddings into scene-wide forecasts using a novel latent context module. This module learns a scene-wide latent space from multiple agent-centric embeddings, enabling joint forecasting and interaction modeling. The competitive performance in the Waymo Open Interaction Prediction Challenge demonstrates the effectiveness of our approach. Moreover, we cluster future waypoints in time and space to quantify the interaction between agents. We merge all modes and analyze each mode independently to determine which clusters are resolved through interaction or result in conflict. Our implementation is available at: https://github.com/kit-mrt/future-motion

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 34 of its 41 references that are in the map. First posted on arXiv 2 Aug 2024.

Builds on

34 of 41 references.

References · 34

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2018Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networkscited by 687
  3. 2021Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Datasetcited by 532
  4. 2020VectorNet: Encoding HD Maps and Agent Dynamics from Vectorized Representationcited by 528
  5. 2022Wayformer: Motion Forecasting via Simple & Efficient Attention Networkscited by 279
  6. 2022Motion Transformer with Global Intention Localization and Local Movement Refinementcited by 267
  7. 2023MotionDiffuser: Controllable Multi-Agent Motion Prediction using Diffusioncited by 210
  8. 2019Multiple Futures Predictioncited by 178
  9. 2020From Goals, Waypoints & Paths To Long Term Human Trajectory Forecastingcited by 166
  10. 2021Scene Transformer: A unified architecture for predicting multiple agent trajectoriescited by 165
  11. 2023MotionLM: Multi-Agent Motion Forecasting as Language Modelingcited by 154
  12. 2023GameFormer: Game-theoretic Modeling and Learning of Transformer-based Interactive Prediction and Planning for Autonomous Drivingcited by 141
22 more
  1. 2021Latent Variable Sequential Set Transformers For Joint Multi-Agent Motion Predictioncited by 131
  2. 2020Implicit Latent Variable Model for Scene-Consistent Motion Forecastingcited by 116
  3. 2021THOMAS: Trajectory Heatmap Output with learned Multi-Agent Samplingcited by 101
  4. 2019Multi-Head Attention for Multi-Modal Joint Vehicle Motion Forecastingcited by 87
  5. 2022M2I: From Factored Marginal Trajectory Prediction to Interactive Predictioncited by 81
  6. 2020Trajectory Forecasts in Unknown Environments Conditioned on Grid-Based Planscited by 72
  7. 2022GoRela: Go Relative for Viewpoint-Invariant Motion Forecastingcited by 68
  8. 2022Occupancy Flow Fields for Motion Forecasting in Autonomous Drivingcited by 62
  9. 2021Identifying Driver Interactions via Conditional Behavior Predictioncited by 48
  10. 2023ProphNet: Efficient Agent-Centric Motion Forecasting with Anchor-Informed Proposalscited by 47
  11. 2023Traj-MAE: Masked Autoencoders for Trajectory Predictioncited by 42
  12. 2022JFP: Joint Future Prediction with Interactive Multi-Agent Modeling for Autonomous Drivingcited by 40
  13. 2023Real-Time Motion Prediction via Heterogeneous Polyline Transformer with Relative Pose Encodingcited by 32
  14. 2023Implicit Occupancy Flow Fields for Perception and Prediction in Self-Drivingcited by 30
  15. 2022PreTraM: Self-Supervised Pre-training via Connecting Trajectory and Mapcited by 25
  16. 2022Interpretable Self-Aware Neural Networks for Robust Trajectory Predictioncited by 22
  17. 2022RAP: Risk-Aware Prediction for Robust Planningcited by 12
  18. 2024JointMotion: Joint Self-Supervision for Joint Motion Predictioncited by 7
  19. 2023RedMotion: Motion Prediction via Redundancy Reductioncited by 6
  20. 2024Learning Online Belief Prediction for Efficient POMDP Planning in Autonomous Drivingcited by 5
  21. 2023Decision-theoretic MPC: Motion Planning with Weighted Maneuver Preferences Under Uncertaintycited by 4
  22. 2024MAP-Former: Multi-Agent-Pair Gaussian Joint Predictioncited by 2

Built on by

Cited by year

Citing papers per year: 2025: 4.

4 papers.

  1. 2025Uncertainty-Aware Trajectory Prediction via Rule-Regularized Heteroscedastic Deep Classificationcited by 3
  2. 2025RetroMotion: Retrocausal Motion Forecasting Models are Instructablecited by 2
  3. 2025Conditional Prediction by Simulation for Automated Drivingcited by 0
  4. 2025From Marginal to Joint Predictions: Evaluating Scene-Consistent Trajectory Prediction Approaches for Automated Drivingcited by 0