mlsanityData to 7 Oct 2026

JointMotion: Joint Self-Supervision for Joint Motion Prediction

arXiv 2403.05489 ↗First posted 8 Mar 2024cs.CV cs.ROcited by 7

Not on the map. Why?

Abstract (arXiv)

We present JointMotion, a self-supervised pre-training method for joint motion prediction in self-driving vehicles. Our method jointly optimizes a scene-level objective connecting motion and environments, and an instance-level objective to refine learned representations. Scene-level representations are learned via non-contrastive similarity learning of past motion sequences and environment context. At the instance level, we use masked autoencoding to refine multimodal polyline representations. We complement this with an adaptive pre-training decoder that enables JointMotion to generalize across different environment representations, fusion mechanisms, and dataset characteristics. Notably, our method reduces the joint final displacement error of Wayformer, HPTR, and Scene Transformer models by 3%, 8%, and 12%, respectively; and enables transfer learning between the Waymo Open Motion and the Argoverse 2 Motion Forecasting datasets. Code: https://github.com/kit-mrt/future-motion

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 41 of its 42 references that are in the map. First posted on arXiv 8 Mar 2024.

Builds on

41 of 42 references.

References · 41

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  5. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  6. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  7. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  8. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  9. 2020Longformer: The Long-Document Transformercited by 3,362
  10. 2021Barlow Twins: Self-Supervised Learning via Redundancy Reductioncited by 1,818
  11. 2023Argoverse 2: Next Generation Datasets for Self-Driving Perception and Forecastingcited by 701
  12. 2021Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Datasetcited by 532
29 more
  1. 2022Masked Autoencoders As Spatiotemporal Learnerscited by 428
  2. 2020Rethinking Pre-training and Self-trainingcited by 352
  3. 2022Wayformer: Motion Forecasting via Simple & Efficient Attention Networkscited by 279
  4. 2022Motion Transformer with Global Intention Localization and Local Movement Refinementcited by 267
  5. 2023A Cookbook of Self-Supervised Learningcited by 248
  6. 2023MotionDiffuser: Controllable Multi-Agent Motion Prediction using Diffusioncited by 210
  7. 2021Scene Transformer: A unified architecture for predicting multiple agent trajectoriescited by 165
  8. 2023MotionLM: Multi-Agent Motion Forecasting as Language Modelingcited by 154
  9. 2019Transformers without Tears: Improving the Normalization of Self-Attentioncited by 153
  10. 2023GameFormer: Game-theoretic Modeling and Learning of Transformer-based Interactive Prediction and Planning for Autonomous Drivingcited by 141
  11. 2023MTR++: Multi-Agent Motion Prediction with Symmetric Scene Modeling and Guided Intention Queryingcited by 139
  12. 2020Implicit Latent Variable Model for Scene-Consistent Motion Forecastingcited by 116
  13. 2021THOMAS: Trajectory Heatmap Output with learned Multi-Agent Samplingcited by 101
  14. 2022M2I: From Factored Marginal Trajectory Prediction to Interactive Predictioncited by 81
  15. 2023Forecast-MAE: Self-supervised Pre-training for Motion Forecasting with Masked Autoencoderscited by 72
  16. 2022GoRela: Go Relative for Viewpoint-Invariant Motion Forecastingcited by 68
  17. 2023ProphNet: Efficient Agent-Centric Motion Forecasting with Anchor-Informed Proposalscited by 47
  18. 2023Traj-MAE: Masked Autoencoders for Trajectory Predictioncited by 42
  19. 2022JFP: Joint Future Prediction with Interactive Multi-Agent Modeling for Autonomous Drivingcited by 40
  20. 2023Real-Time Motion Prediction via Heterogeneous Polyline Transformer with Relative Pose Encodingcited by 32
  21. 2022SSL-Lanes: Self-Supervised Learning for Motion Forecasting in Autonomous Drivingcited by 31
  22. 2023QCNeXt: A Next-Generation Framework For Joint Multi-Agent Trajectory Predictioncited by 31
  23. 2022PreTraM: Self-Supervised Pre-training via Connecting Trajectory and Mapcited by 25
  24. 2023Improved baselines for vision-language pre-trainingcited by 20
  25. 2022MPA: MultiPath++ Based Architecture for Motion Predictioncited by 13
  26. 2022RAP: Risk-Aware Prediction for Robust Planningcited by 12
  27. 2023RedMotion: Motion Prediction via Redundancy Reductioncited by 6
  28. 2023RMP: A Random Mask Pretrain Framework for Motion Predictioncited by 6
  29. 2023Decision-theoretic MPC: Motion Planning with Weighted Maneuver Preferences Under Uncertaintycited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 3; 2026: 2.

7 papers.

  1. 2024EMMA: End-to-End Multimodal Model for Autonomous Drivingcited by 226
  2. 2025Motion Forecasting for Autonomous Vehicles: A Surveycited by 8
  3. 2024SceneMotion: From Agent-Centric Embeddings to Scene-Wide Forecastscited by 4
  4. 2025Generative AI for Autonomous Driving: A Reviewcited by 3
  5. 2025RetroMotion: Retrocausal Motion Forecasting Models are Instructablecited by 2
  6. 2026Causality-Aware End-to-End Autonomous Driving via Ego-Centric Joint Scene Modelingcited by 1
  7. 2026Forecasting the Past: Gradient-Based Distribution Shift Detection in Trajectory Predictioncited by 0