# SSL-Lanes: Self-Supervised Learning for Motion Forecasting in Autonomous Driving

arXiv: https://arxiv.org/abs/2206.14116 · HTML: https://mlsanity.com/paper/2206.14116/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- Builds on 46 of its 54 references that are in the map.
- First posted on arXiv 28 Jun 2022.

## Builds on (most cited first, 30 of 46)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [Similarity of Neural Network Representations Revisited](https://mlsanity.com/paper/1905.00414/) (2019)
- [Unsupervised Representation Learning by Predicting Image Rotations](https://mlsanity.com/paper/1803.07728/) (2018)
- [Context Encoders: Feature Learning by Inpainting](https://mlsanity.com/paper/1604.07379/) (2016)
- [Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles](https://mlsanity.com/paper/1603.09246/) (2016)
- [Objects as Points](https://mlsanity.com/paper/1904.07850/) (2019)
- [Colorful Image Colorization](https://mlsanity.com/paper/1603.08511/) (2016)
- [Deeper Insights into Graph Convolutional Networks for Semi-Supervised Learning](https://mlsanity.com/paper/1801.07606/) (2018)
- [Unsupervised Visual Representation Learning by Context Prediction](https://mlsanity.com/paper/1505.05192/) (2015)
- [Strategies for Pre-training Graph Neural Networks](https://mlsanity.com/paper/1905.12265/) (2019)
- [Argoverse: 3D Tracking and Forecasting with Rich Maps](https://mlsanity.com/paper/1911.02620/) (2019)
- [Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Dataset](https://mlsanity.com/paper/2104.10133/) (2021)
- [VectorNet: Encoding HD Maps and Agent Dynamics from Vectorized Representation](https://mlsanity.com/paper/2005.04259/) (2020)
- [MultiPath: Multiple Probabilistic Anchor Trajectory Hypotheses for Behavior Prediction](https://mlsanity.com/paper/1910.05449/) (2019)
- [ChauffeurNet: Learning to Drive by Imitating the Best and Synthesizing the Worst](https://mlsanity.com/paper/1812.03079/) (2018)
- [Learning Lane Graph Representations for Motion Forecasting](https://mlsanity.com/paper/2007.13732/) (2020)
- [TNT: Target-driveN Trajectory Prediction](https://mlsanity.com/paper/2008.08294/) (2020)
- [Social-STGCNN: A Social Spatio-Temporal Graph Convolutional Neural Network for Human Trajectory Prediction](https://mlsanity.com/paper/2002.11927/) (2020)
- [DenseTNT: End-to-end Trajectory Prediction from Dense Goal Sets](https://mlsanity.com/paper/2108.09640/) (2021)
- [Graph Self-Supervised Learning: A Survey](https://mlsanity.com/paper/2103.00111/) (2021)
- [CoverNet: Multimodal Behavior Prediction using Trajectory Sets](https://mlsanity.com/paper/1911.10298/) (2019)
- [MultiPath++: Efficient Information Fusion and Trajectory Aggregation for Behavior Prediction](https://mlsanity.com/paper/2111.14973/) (2021)
- [End-to-end Interpretable Neural Motion Planner](https://mlsanity.com/paper/2101.06679/) (2021)
- [Multimodal Motion Prediction with Stacked Transformers](https://mlsanity.com/paper/2103.11624/) (2021)
- [IntentNet: Learning to Predict Intention from Raw Sensor Data](https://mlsanity.com/paper/2101.07907/) (2021)

## Built on by (most cited first, 30 of 32)

- [Forecast-MAE: Self-supervised Pre-training for Motion Forecasting with Masked Autoencoders](https://mlsanity.com/paper/2308.09882/) (2023)
- [A Clinical Benchmark of Public Self-Supervised Pathology Foundation Models](https://mlsanity.com/paper/2407.06508/) (2024)
- [UniTraj: A Unified Framework for Scalable Vehicle Trajectory Prediction](https://mlsanity.com/paper/2403.15098/) (2024)
- [FJMP: Factorized Joint Multi-Agent Motion Prediction over Learned Directed Acyclic Interaction Graphs](https://mlsanity.com/paper/2211.16197/) (2022)
- [BAT: Behavior-Aware Human-Like Trajectory Prediction for Autonomous Driving](https://mlsanity.com/paper/2312.06371/) (2023)
- [SIMPL: A Simple and Efficient Multi-agent Motion Prediction Baseline for Autonomous Driving](https://mlsanity.com/paper/2402.02519/) (2024)
- [Pre-training on Synthetic Driving Data for Trajectory Prediction](https://mlsanity.com/paper/2309.10121/) (2023)
- [Motion Style Transfer: Modular Low-Rank Adaptation for Deep Motion Forecasting](https://mlsanity.com/paper/2211.03165/) (2022)
- [MFTraj: Map-Free, Behavior-Driven Trajectory Prediction for Autonomous Driving](https://mlsanity.com/paper/2405.01266/) (2024)
- [Exploiting map information for self-supervised learning in motion forecasting](https://mlsanity.com/paper/2210.04672/) (2022)
- [Motion Forecasting for Autonomous Vehicles: A Survey](https://mlsanity.com/paper/2502.08664/) (2025)
- [JointMotion: Joint Self-Supervision for Joint Motion Prediction](https://mlsanity.com/paper/2403.05489/) (2024)
- [SmartPretrain: Model-Agnostic and Dataset-Agnostic Representation Learning for Motion Prediction](https://mlsanity.com/paper/2410.08669/) (2024)
- [RMP: A Random Mask Pretrain Framework for Motion Prediction](https://mlsanity.com/paper/2309.08989/) (2023)
- [FutureNet-LOF: Joint Trajectory Prediction and Lane Occupancy Field Prediction with Future Context Encoding](https://mlsanity.com/paper/2406.14422/) (2024)
- [iPLAN: Intent-Aware Planning in Heterogeneous Traffic via Distributed Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2306.06236/) (2023)
- [Improving Autonomous Driving Safety with POP: A Framework for Accurate Partially Observed Trajectory Predictions](https://mlsanity.com/paper/2309.15685/) (2023)
- [Future-Aware Interaction Network For Motion Forecasting](https://mlsanity.com/paper/2503.06565/) (2025)
- [Forecast-PEFT: Parameter-Efficient Fine-Tuning for Pre-trained Motion Forecasting Models](https://mlsanity.com/paper/2407.19564/) (2024)
- [Multi-scale Temporal Fusion Transformer for Incomplete Vehicle Trajectory Prediction](https://mlsanity.com/paper/2409.00904/) (2024)
- [RealTraj: Towards Real-World Pedestrian Trajectory Forecasting](https://mlsanity.com/paper/2411.17376/) (2024)
- [Towards Generalizable Trajectory Prediction Using Dual-Level Representation Learning And Adaptive Prompting](https://mlsanity.com/paper/2501.04815/) (2025)
- [OmniTraj: Pre-Training on Heterogeneous Data for Adaptive and Zero-Shot Human Trajectory Prediction](https://mlsanity.com/paper/2507.23657/) (2025)
- [ADM: Accelerated Diffusion Model via Estimated Priors for Robust Motion Prediction under Uncertainties](https://mlsanity.com/paper/2405.00797/) (2024)
- [A Structure-Aware Lane Graph Transformer Model for Vehicle Trajectory Prediction](https://mlsanity.com/paper/2405.20121/) (2024)
- [PPT: Pretraining with Pseudo-Labeled Trajectories for Motion Forecasting](https://mlsanity.com/paper/2412.06491/) (2024)
- [Enhancing Trajectory Prediction through Self-Supervised Waypoint Noise Prediction](https://mlsanity.com/paper/2312.09466/) (2023)
- [Interpretable Interaction Modeling for Trajectory Prediction via Agent Selection and Physical Coefficient](https://mlsanity.com/paper/2405.13152/) (2024)
- [Noise-Free Explanation for Driving Action Prediction](https://mlsanity.com/paper/2407.06339/) (2024)
- [Driving behavior recognition via self-discovery learning](https://mlsanity.com/paper/2503.14194/) (2025)

## Abstract (arXiv)

Self-supervised learning (SSL) is an emerging technique that has been successfully employed to train convolutional neural networks (CNNs) and graph neural networks (GNNs) for more transferable, generalizable, and robust representation learning. However its potential in motion forecasting for autonomous driving has rarely been explored. In this study, we report the first systematic exploration and assessment of incorporating self-supervision into motion forecasting. We first propose to investigate four novel self-supervised learning tasks for motion forecasting with theoretical rationale and quantitative and qualitative comparisons on the challenging large-scale Argoverse dataset. Secondly, we point out that our auxiliary SSL-based learning setup not only outperforms forecasting methods which use transformers, complicated fusion mechanisms and sophisticated online dense goal candidate optimization algorithms in terms of performance accuracy, but also has low inference time and architectural complexity. Lastly, we conduct several experiments to understand why SSL improves motion forecasting. Code is open-sourced at urlhttps://github.com/AutoVision-cloud/SSL-Lanes.