mlsanityData to 7 Oct 2026

RMP: A Random Mask Pretrain Framework for Motion Prediction

arXiv 2309.08989 ↗First posted 16 Sep 2023cs.RO cs.CV cs.LGcited by 6

Not on the map. Why?

Abstract (arXiv)

As the pretraining technique is growing in popularity, little work has been done on pretrained learning-based motion prediction methods in autonomous driving. In this paper, we propose a framework to formalize the pretraining task for trajectory prediction of traffic participants. Within our framework, inspired by the random masked model in natural language processing (NLP) and computer vision (CV), objects' positions at random timesteps are masked and then filled in by the learned neural network (NN). By changing the mask profile, our framework can easily switch among a range of motion-related tasks. We show that our proposed pretraining framework is able to deal with noisy inputs and improves the motion prediction accuracy and miss rate, especially for objects occluded over time by evaluating it on Argoverse and NuScenes datasets.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 38 of its 44 references that are in the map. First posted on arXiv 16 Sep 2023.

Builds on

38 of 44 references.

References · 38

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  4. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  5. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  6. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  7. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,845
  8. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,360
  9. 2022VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Trainingcited by 1,305
  10. 2018Learning deep representations by mutual information estimation and maximizationcited by 1,265
  11. 2019Argoverse: 3D Tracking and Forecasting with Rich Mapscited by 728
  12. 2021Point-BERT: Pre-training 3D Point Cloud Transformers with Masked Point Modelingcited by 647
26 more
  1. 2021Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Datasetcited by 532
  2. 2020VectorNet: Encoding HD Maps and Agent Dynamics from Vectorized Representationcited by 528
  3. 2022Masked Autoencoders As Spatiotemporal Learnerscited by 428
  4. 2019MultiPath: Multiple Probabilistic Anchor Trajectory Hypotheses for Behavior Predictioncited by 417
  5. 2018ChauffeurNet: Learning to Drive by Imitating the Best and Synthesizing the Worstcited by 391
  6. 2020Learning Lane Graph Representations for Motion Forecastingcited by 380
  7. 2021DenseTNT: End-to-end Trajectory Prediction from Dense Goal Setscited by 292
  8. 2021MultiPath++: Efficient Information Fusion and Trajectory Aggregation for Behavior Predictioncited by 246
  9. 2019INTERACTION Dataset: An INTERnational, Adversarial and Cooperative moTION Dataset in Interactive Driving Scenarios with Semantic Mapscited by 231
  10. 2021Multimodal Motion Prediction with Stacked Transformerscited by 207
  11. 2022Masked World Models for Visual Controlcited by 169
  12. 2021GOHOME: Graph-Oriented Heatmap Output for future Motion Estimationcited by 146
  13. 2021Latent Variable Sequential Set Transformers For Joint Multi-Agent Motion Predictioncited by 131
  14. 2021HOME: Heatmap Output for future Motion Estimationcited by 122
  15. 2019Rules of the Road: Predicting Driving Behavior with a Convolutional Model of Semantic Interactionscited by 117
  16. 2021Multimodal Trajectory Prediction Conditioned on Lane-Graph Traversalscited by 112
  17. 2021THOMAS: Trajectory Heatmap Output with learned Multi-Agent Samplingcited by 101
  18. 2020What-If Motion Prediction for Autonomous Drivingcited by 80
  19. 2021Multi-modal Motion Prediction with Transformer-based Neural Network for Autonomous Drivingcited by 73
  20. 2023Leveraging Future Relationship Reasoning for Vehicle Trajectory Predictioncited by 47
  21. 2023Traj-MAE: Masked Autoencoders for Trajectory Predictioncited by 42
  22. 2022SSL-Lanes: Self-Supervised Learning for Motion Forecasting in Autonomous Drivingcited by 31
  23. 2022PreTraM: Self-Supervised Pre-training via Connecting Trajectory and Mapcited by 25
  24. 2021Multi-Agent Variational Occlusion Inference Using People as Sensorscited by 10
  25. 2022Exploiting map information for self-supervised learning in motion forecastingcited by 8
  26. 2022LOPR: Latent Occupancy PRediction using Generative Modelscited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 2; 2025: 1; 2026: 2.

6 papers.

  1. 2023RealGen: Retrieval Augmented Generation for Controllable Traffic Scenarioscited by 46
  2. 2024JointMotion: Joint Self-Supervision for Joint Motion Predictioncited by 7
  3. 2024SmartPretrain: Model-Agnostic and Dataset-Agnostic Representation Learning for Motion Predictioncited by 7
  4. 2026A Generalizable Physics-guided Causal Model for Trajectory Prediction in Autonomous Drivingcited by 1
  5. 2025Ensemble of Pre-Trained Models for Long-Tailed Trajectory Predictioncited by 0
  6. 2026Hope for the Best, Prepare for the Worst: Occlusion-Aware Contingency Planning for Autonomous Vehiclescited by 0