mlsanityData to 14 Oct 2026

SEPT: Towards Efficient Scene Representation Learning for Motion Prediction

arXiv 2309.15289 ↗First posted 26 Sep 2023cs.CV cs.LGcited by 27

Not on the map. Why?

Abstract (arXiv)

Motion prediction is crucial for autonomous vehicles to operate safely in complex traffic environments. Extracting effective spatiotemporal relationships among traffic elements is key to accurate forecasting. Inspired by the successful practice of pretrained large language models, this paper presents SEPT, a modeling framework that leverages self-supervised learning to develop powerful spatiotemporal understanding for complex traffic scenes. Specifically, our approach involves three masking-reconstruction modeling tasks on scene inputs including agents' trajectories and road network, pretraining the scene encoder to capture kinematics within trajectory, spatial structure of road network, and interactions among roads and agents. The pretrained encoder is then finetuned on the downstream forecasting task. Extensive experiments demonstrate that SEPT, without elaborate architectural design or manual feature engineering, achieves state-of-the-art performance on the Argoverse 1 and Argoverse 2 motion forecasting benchmarks, outperforming previous methods on all main metrics by a large margin.

Key facts. Cited by 27 papers in the map (data through 14 Oct 2026). Builds on 26 of its 33 references that are in the map. First posted on arXiv 26 Sep 2023.

Builds on

26 of 33 references.

References · 26

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  3. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  4. 2020VectorNet: Encoding HD Maps and Agent Dynamics from Vectorized Representationcited by 530
  5. 2019MultiPath: Multiple Probabilistic Anchor Trajectory Hypotheses for Behavior Predictioncited by 420
  6. 2020Learning Lane Graph Representations for Motion Forecastingcited by 382
  7. 2017DESIRE: Distant Future Prediction in Dynamic Scenes with Interacting Agentscited by 310
  8. 2021DenseTNT: End-to-end Trajectory Prediction from Dense Goal Setscited by 295
  9. 2022Wayformer: Motion Forecasting via Simple & Efficient Attention Networkscited by 280
  10. 2022Motion Transformer with Global Intention Localization and Local Movement Refinementcited by 270
  11. 2019CoverNet: Multimodal Behavior Prediction using Trajectory Setscited by 259
  12. 2021MultiPath++: Efficient Information Fusion and Trajectory Aggregation for Behavior Predictioncited by 246
14 more
  1. 2019Multiple Futures Predictioncited by 178
  2. 2021Scene Transformer: A unified architecture for predicting multiple agent trajectoriescited by 165
  3. 2021HOME: Heatmap Output for future Motion Estimationcited by 122
  4. 2022HDGT: Heterogeneous Driving Graph Transformer for Multi-Agent Trajectory Prediction via Scene Encodingcited by 110
  5. 2019Multi-Head Attention for Multi-Modal Joint Vehicle Motion Forecastingcited by 87
  6. 2021Multi-modal Motion Prediction with Transformer-based Neural Network for Autonomous Drivingcited by 73
  7. 2023Forecast-MAE: Self-supervised Pre-training for Motion Forecasting with Masked Autoencoderscited by 73
  8. 2022GANet: Goal Area Network for Motion Forecastingcited by 49
  9. 2023ProphNet: Efficient Agent-Centric Motion Forecasting with Anchor-Informed Proposalscited by 48
  10. 2023Traj-MAE: Masked Autoencoders for Trajectory Predictioncited by 42
  11. 2023MacFormer: Map-Agent Coupled Transformer for Real-time and Robust Trajectory Predictioncited by 32
  12. 2023Dynamic Scenario Representation Learning for Motion Forecasting with Heterogeneous Graph Convolutional Recurrent Networkscited by 28
  13. 2021Integrated Decision and Control: Towards Interpretable and Computationally Efficient Driving Intelligencecited by 27
  14. 2022Bootstrap Motion Forecasting With Self-Consistent Constraintscited by 23

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 9; 2025: 11; 2026: 7.

27 papers.

Sort:

By area:

  1. 2024Optimizing Diffusion Models for Joint Trajectory Prediction and Controllable Generationcited by 18
  2. 2024Motion Forecasting in Continuous Drivingcited by 11
  3. 2024Efficient Motion Prediction: A Lightweight & Accurate Trajectory Prediction Model With Fast Training and Inference Speedcited by 10
  4. 2025Multi-modal Knowledge Distillation-based Human Trajectory Forecastingcited by 10
  5. 2025Motion Forecasting for Autonomous Vehicles: A Surveycited by 8
  6. 2024Risk-Aware Vehicle Trajectory Prediction Under Safety-Critical Scenarioscited by 7
  7. 2024SmartPretrain: Model-Agnostic and Dataset-Agnostic Representation Learning for Motion Predictioncited by 7
  8. 2025DONUT: A Decoder-Only Model for Trajectory Predictioncited by 7
  9. 2025GoIRL: Graph-Oriented Inverse Reinforcement Learning for Multimodal Trajectory Predictioncited by 7
  10. 2025Generative Active Learning for Long-tail Trajectory Prediction via Controllable Diffusion Modelcited by 6
  11. 2024Forecast-PEFT: Parameter-Efficient Fine-Tuning for Pre-trained Motion Forecasting Modelscited by 3
  12. 2024Snapshot: Towards Application-centered Models for Pedestrian Trajectory Prediction in Urban Traffic Environmentscited by 3
15 more
  1. 2025Towards Generalizable Trajectory Prediction Using Dual-Level Representation Learning And Adaptive Promptingcited by 3
  2. 2025OmniTraj: Pre-Training on Heterogeneous Data for Adaptive and Zero-Shot Human Trajectory Predictioncited by 3
  3. 2025RetroMotion: Retrocausal Motion Forecasting Models are Instructablecited by 2
  4. 2026Recover to Predict: Progressive Retrospective Learning for Variable-Length Trajectory Predictioncited by 2
  5. 2025Structure-preserving contrastive learning for spatial time seriescited by 1
  6. 2025LMFormer: Lane based Motion Prediction Transformercited by 1
  7. 2026Streaming Real-Time Trajectory Prediction Using Endpoint-Aware Modelingcited by 1
  8. 2024Efficient and Interpretable Traffic Destination Prediction using Explainable Boosting Machinescited by 0
  9. 2024Reinforced Imitative Trajectory Planning for Urban Automated Drivingcited by 0
  10. 2025Ensemble of Pre-Trained Models for Long-Tailed Trajectory Predictioncited by 0
  11. 2026TaPD: Temporal-adaptive Progressive Distillation for Observation-Adaptive Trajectory Forecasting in Autonomous Drivingcited by 0
  12. 2026SHARP: Short-Window Streaming for Accurate and Robust Prediction in Motion Forecastingcited by 0
  13. 2026Mind the Noise: Sensitivity of Transformer-based Interaction-Aware Trajectory Prediction Models to Noisy Datacited by 0
  14. 2026Towards Metric-Agnostic Trajectory Forecastingcited by 0
  15. 2026IMR: Iterative Mode-World Weighted Regression for Multi-Agent Trajectory Predictioncited by 0