mlsanityData to 14 Oct 2026

Forecast-MAE: Self-supervised Pre-training for Motion Forecasting with Masked Autoencoders

arXiv 2308.09882 ↗First posted 19 Aug 2023cs.RO cs.CVcited by 73

Not on the map. Why?

Abstract (arXiv)

This study explores the application of self-supervised learning (SSL) to the task of motion forecasting, an area that has not yet been extensively investigated despite the widespread success of SSL in computer vision and natural language processing. To address this gap, we introduce Forecast-MAE, an extension of the mask autoencoders framework that is specifically designed for self-supervised learning of the motion forecasting task. Our approach includes a novel masking strategy that leverages the strong interconnections between agents' trajectories and road networks, involving complementary masking of agents' future or history trajectories and random masking of lane segments. Our experiments on the challenging Argoverse 2 motion forecasting benchmark show that Forecast-MAE, which utilizes standard Transformer blocks with minimal inductive bias, achieves competitive performance compared to state-of-the-art methods that rely on supervised learning and sophisticated designs. Moreover, it outperforms the previous self-supervised learning method by a significant margin. Code is available at https://github.com/jchengai/forecast-mae.

Key facts. Cited by 73 papers in the map (data through 14 Oct 2026). Builds on 46 of its 57 references that are in the map. First posted on arXiv 19 Aug 2023.

Builds on

46 of 57 references.

References · 46

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  5. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  6. 2020End-to-End Object Detection with Transformerscited by 7,414
  7. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  8. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  9. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,248
  10. 2016Feature Pyramid Networks for Object Detectioncited by 5,036
  11. 2019Argoverse: 3D Tracking and Forecasting with Rich Mapscited by 732
  12. 2023Argoverse 2: Next Generation Datasets for Self-Driving Perception and Forecastingcited by 707
34 more
  1. 2021Point-BERT: Pre-training 3D Point Cloud Transformers with Masked Point Modelingcited by 653
  2. 2021Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Datasetcited by 537
  3. 2020VectorNet: Encoding HD Maps and Agent Dynamics from Vectorized Representationcited by 530
  4. 2022Masked Autoencoders for Point Cloud Self-supervised Learningcited by 459
  5. 2019MultiPath: Multiple Probabilistic Anchor Trajectory Hypotheses for Behavior Predictioncited by 420
  6. 2020Learning Lane Graph Representations for Motion Forecastingcited by 382
  7. 2020TNT: Target-driveN Trajectory Predictioncited by 347
  8. 2021DenseTNT: End-to-end Trajectory Prediction from Dense Goal Setscited by 295
  9. 2022Masked Autoencoders that Listencited by 273
  10. 2022Motion Transformer with Global Intention Localization and Local Movement Refinementcited by 270
  11. 2019CoverNet: Multimodal Behavior Prediction using Trajectory Setscited by 259
  12. 2021MultiPath++: Efficient Information Fusion and Trajectory Aggregation for Behavior Predictioncited by 246
  13. 2022Neighborhood Attention Transformercited by 239
  14. 2021Multimodal Motion Prediction with Stacked Transformerscited by 208
  15. 2019Multiple Futures Predictioncited by 178
  16. 2021GOHOME: Graph-Oriented Heatmap Output for future Motion Estimationcited by 146
  17. 2021Latent Variable Sequential Set Transformers For Joint Multi-Agent Motion Predictioncited by 131
  18. 2021HOME: Heatmap Output for future Motion Estimationcited by 122
  19. 2021LaneRCNN: Distributed Representations for Graph-Centric Motion Forecastingcited by 113
  20. 2021Multimodal Trajectory Prediction Conditioned on Lane-Graph Traversalscited by 113
  21. 2022HDGT: Heterogeneous Driving Graph Transformer for Multi-Agent Trajectory Prediction via Scene Encodingcited by 110
  22. 2021THOMAS: Trajectory Heatmap Output with learned Multi-Agent Samplingcited by 101
  23. 2021TPCN: Temporal Point Cloud Networks for Motion Forecastingcited by 94
  24. 2020What-If Motion Prediction for Autonomous Drivingcited by 80
  25. 2022GoRela: Go Relative for Viewpoint-Invariant Motion Forecastingcited by 68
  26. 2021Learning to Predict Vehicle Trajectories with Model-based Planningcited by 63
  27. 2022GANet: Goal Area Network for Motion Forecastingcited by 49
  28. 2022SSL-Lanes: Self-Supervised Learning for Motion Forecasting in Autonomous Drivingcited by 32
  29. 2022PreTraM: Self-Supervised Pre-training via Connecting Trajectory and Mapcited by 25
  30. 2021AVGCN: Trajectory Prediction using Graph Convolutional Networks Guided by Human Attentioncited by 12
  31. 2022BANet: Motion Forecasting with Boundary Aware Networkcited by 10
  32. 2022QML for Argoverse 2 Motion Forecasting Challengecited by 8
  33. 2020HGCN-GJS: Hierarchical Graph Convolutional Network with Groupwise Joint Sampling for Trajectory Predictioncited by 4
  34. 2020CoMoGCN: Coherent Motion Aware Trajectory Prediction with Graph Representationcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 7; 2024: 30; 2025: 20; 2026: 16.

73 papers.

Sort:

By area:

  1. 2024textitS3Gaussian: Self-Supervised Street Gaussians for Autonomous Drivingcited by 105
  2. 2024PLUTO: Pushing the Limit of Imitation Learning-based Planning for Autonomous Drivingcited by 84
  3. 2023RealGen: Retrieval Augmented Generation for Controllable Traffic Scenarioscited by 46
  4. 2024HPNet: Dynamic Trajectory Forecasting with Historical Prediction Attentioncited by 38
  5. 2024DrivingRecon: Large 4D Gaussian Reconstruction Model For Autonomous Drivingcited by 29
  6. 2023SEPT: Towards Efficient Scene Representation Learning for Motion Predictioncited by 27
  7. 2024SIMPL: A Simple and Efficient Multi-agent Motion Prediction Baseline for Autonomous Drivingcited by 27
  8. 2023Rethinking Imitation-based Planner for Autonomous Drivingcited by 25
  9. 2024T4P: Test-Time Training of Trajectory Prediction via Masked Autoencoder and Actor-specific Token Memorycited by 20
  10. 2023Pre-training on Synthetic Driving Data for Trajectory Predictioncited by 18
  11. 2024Optimizing Diffusion Models for Joint Trajectory Prediction and Controllable Generationcited by 18
  12. 2023Masked Modeling for Self-supervised Representation Learning on Vision and Beyondcited by 13
38 more
  1. 2024Instruct Large Language Models to Drive like Humanscited by 11
  2. 2024CooPre: Cooperative Pretraining for V2X Cooperative Perceptioncited by 11
  3. 2024Motion Forecasting in Continuous Drivingcited by 11
  4. 2024Efficient Motion Prediction: A Lightweight & Accurate Trajectory Prediction Model With Fast Training and Inference Speedcited by 10
  5. 2025Multi-modal Knowledge Distillation-based Human Trajectory Forecastingcited by 10
  6. 2024Improving Out-of-Distribution Generalization of Trajectory Prediction for Autonomous Driving via Polynomial Representationscited by 9
  7. 2024Learning Multiple Probabilistic Decisions from Latent World Model in Autonomous Drivingcited by 8
  8. 2025Motion Forecasting for Autonomous Vehicles: A Surveycited by 8
  9. 2024JointMotion: Joint Self-Supervision for Joint Motion Predictioncited by 7
  10. 2024SmartPretrain: Model-Agnostic and Dataset-Agnostic Representation Learning for Motion Predictioncited by 7
  11. 2025DONUT: A Decoder-Only Model for Trajectory Predictioncited by 7
  12. 2024Quantifying Uncertainty in Motion Prediction with Variational Bayesian Mixturecited by 6
  13. 2024FutureNet-LOF: Joint Trajectory Prediction and Lane Occupancy Field Prediction with Future Context Encodingcited by 6
  14. 2025Generative Active Learning for Long-tail Trajectory Prediction via Controllable Diffusion Modelcited by 6
  15. 2023Improving Autonomous Driving Safety with POP: A Framework for Accurate Partially Observed Trajectory Predictionscited by 5
  16. 2025B'ezierGS: Dynamic Urban Scene Reconstruction with B'ezier Curve Gaussian Splattingcited by 5
  17. 2025Interaction-Merged Motion Planning: Effectively Leveraging Diverse Motion Datasets for Robust Planningcited by 5
  18. 2024Street Gaussians without 3D Object Trackercited by 4
  19. 2025Future-Aware Interaction Network For Motion Forecastingcited by 4
  20. 2024Forecast-PEFT: Parameter-Efficient Fine-Tuning for Pre-trained Motion Forecasting Modelscited by 3
  21. 2024Snapshot: Towards Application-centered Models for Pedestrian Trajectory Prediction in Urban Traffic Environmentscited by 3
  22. 2024RealTraj: Towards Real-World Pedestrian Trajectory Forecastingcited by 3
  23. 2025Towards Generalizable Trajectory Prediction Using Dual-Level Representation Learning And Adaptive Promptingcited by 3
  24. 2025OmniTraj: Pre-Training on Heterogeneous Data for Adaptive and Zero-Shot Human Trajectory Predictioncited by 3
  25. 2025Foundation Models for Trajectory Planning in Autonomous Driving: A Review of Progress and Open Challengescited by 3
  26. 2024PPT: Pretraining with Pseudo-Labeled Trajectories for Motion Forecastingcited by 2
  27. 2024ITPNet: Towards Instantaneous Trajectory Prediction for Autonomous Drivingcited by 2
  28. 2025HAMF: A Hybrid Attention-Mamba Framework for Joint Scene Context Understanding and Future Motion Representation Learningcited by 2
  29. 2025RetroMotion: Retrocausal Motion Forecasting Models are Instructablecited by 2
  30. 2025Towards Predicting Any Human Trajectory In Contextcited by 2
  31. 2025Deep Space Weather Model: Long-Range Solar Flare Prediction from Multi-Wavelength Imagescited by 2
  32. 2026Recover to Predict: Progressive Retrospective Learning for Variable-Length Trajectory Predictioncited by 2
  33. 2026CorrectionPlanner: Self-Correction Planner with Reinforcement Learning in Autonomous Drivingcited by 2
  34. 2024PreGSU-A Generalized Traffic Scene Understanding Model for Autonomous Driving based on Pre-trained Graph Attention Networkcited by 1
  35. 2024Hybrid Imitation-Learning Motion Planner for Urban Drivingcited by 1
  36. 2024Map-Free Trajectory Prediction with Map Distillation and Hierarchical Encodingcited by 1
  37. 2025Structure-preserving contrastive learning for spatial time seriescited by 1
  38. 2025EP-Diffuser: An Efficient Diffusion Model for Traffic Scene Generation and Prediction via Polynomial Representationscited by 1

50 most cited of 73 All 73 (JSON)