mlsanityData to 14 Oct 2026

Wayformer: Motion Forecasting via Simple & Efficient Attention Networks

arXiv 2207.05844 ↗First posted 12 Jul 2022cs.CVcited by 280

On the map: World models → World models for autonomous driving

Abstract (arXiv)

Motion forecasting for autonomous driving is a challenging task because complex driving scenarios result in a heterogeneous mix of static and dynamic inputs. It is an open problem how best to represent and fuse information about road geometry, lane connectivity, time-varying traffic light state, and history of a dynamic set of agents and their interactions into an effective encoding. To model this diverse set of input features, many approaches proposed to design an equally complex system with a diverse set of modality specific modules. This results in systems that are difficult to scale, extend, or tune in rigorous ways to trade off quality and efficiency. In this paper, we present Wayformer, a family of attention based architectures for motion forecasting that are simple and homogeneous. Wayformer offers a compact model description consisting of an attention based scene encoder and a decoder. In the scene encoder we study the choice of early, late and hierarchical fusion of the input modalities. For each fusion type we explore strategies to tradeoff efficiency and quality via factorized attention or latent query attention. We show that early fusion, despite its simplicity of construction, is not only modality agnostic but also achieves state-of-the-art results on both Waymo Open MotionDataset (WOMD) and Argoverse leaderboards, demonstrating the effectiveness of our design philosophy

Key facts. Cited by 280 papers in the map (data through 14 Oct 2026). Placed in World models for autonomous driving, one of 11 directions in World models. Builds on 48 of its 58 references that are in the map. First posted on arXiv 12 Jul 2022.

Builds on

48 of 58 references.

Closest ancestors · 2

Same direction, newest first

  1. 2020One Thousand and One Hours: Self-driving Motion Prediction Dataset25 Jun
  2. 2018Multimodal Trajectory Predictions for Autonomous Driving using Deep Convolutional Networks18 Sep

Other references · 46

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  5. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  6. 2020End-to-End Object Detection with Transformerscited by 7,414
  7. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  8. 2020Scaling Laws for Neural Language Modelscited by 6,586
  9. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
  10. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,863
  11. 2021ViViT: A Video Vision Transformercited by 1,620
  12. 2021Perceiver: General Perception with Iterative Attentioncited by 1,104
34 more
  1. 2020Efficient Transformers: A Surveycited by 790
  2. 2019Argoverse: 3D Tracking and Forecasting with Rich Mapscited by 732
  3. 2020Long Range Arena: A Benchmark for Efficient Transformerscited by 700
  4. 2018Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networkscited by 690
  5. 2021AST: Audio Spectrogram Transformercited by 643
  6. 2021Perceiver IO: A General Architecture for Structured Inputs & Outputscited by 629
  7. 2019Stand-Alone Self-Attention in Vision Modelscited by 542
  8. 2021Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Datasetcited by 537
  9. 2020VectorNet: Encoding HD Maps and Agent Dynamics from Vectorized Representationcited by 530
  10. 2019MultiPath: Multiple Probabilistic Anchor Trajectory Hypotheses for Behavior Predictioncited by 420
  11. 2020Learning Lane Graph Representations for Motion Forecastingcited by 382
  12. 2021Bottleneck Transformers for Visual Recognitioncited by 364
  13. 2019Attention Augmented Convolutional Networkscited by 363
  14. 2019Axial Attention in Multidimensional Transformerscited by 358
  15. 2020TNT: Target-driveN Trajectory Predictioncited by 347
  16. 2021AgentFormer: Agent-Aware Transformers for Socio-Temporal Multi-Agent Forecastingcited by 331
  17. 2017DESIRE: Distant Future Prediction in Dynamic Scenes with Interacting Agentscited by 310
  18. 2021DenseTNT: End-to-end Trajectory Prediction from Dense Goal Setscited by 295
  19. 2021MultiPath++: Efficient Information Fusion and Trajectory Aggregation for Behavior Predictioncited by 246
  20. 2021End-to-end Interpretable Neural Motion Plannercited by 242
  21. 2019Social-BiGAT: Multimodal Trajectory Forecasting using Bicycle-GAN and Graph Attention Networkscited by 229
  22. 2020Spatio-Temporal Graph Transformer Networks for Pedestrian Trajectory Predictioncited by 218
  23. 2021Multimodal Motion Prediction with Stacked Transformerscited by 208
  24. 2019PRECOG: PREdiction Conditioned On Goals in Visual Multi-Agent Settingscited by 179
  25. 2019Multiple Futures Predictioncited by 178
  26. 2021IntentNet: Learning to Predict Intention from Raw Sensor Datacited by 158
  27. 2019Rules of the Road: Predicting Driving Behavior with a Convolutional Model of Semantic Interactionscited by 117
  28. 2021LaneRCNN: Distributed Representations for Graph-Centric Motion Forecastingcited by 113
  29. 2019Spatially-Aware Graph Neural Networks for Relational Behavior Forecasting from Sensor Datacited by 104
  30. 2019Multi-Head Attention for Multi-Modal Joint Vehicle Motion Forecastingcited by 87
  31. 2020What-If Motion Prediction for Autonomous Drivingcited by 80
  32. 2019The Garden of Forking Paths: Towards Multi-Future Trajectory Predictioncited by 72
  33. 2020Trajectron++: Dynamically-Feasible Trajectory Forecasting With Heterogeneous Datacited by 58
  34. 2020SoDA: Multi-Object Tracking with Soft Data Associationcited by 12

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 13; 2023: 52; 2024: 88; 2025: 68; 2026: 59.

280 papers.

Sort:

By area:

  1. 2022Planning-oriented Autonomous Drivingcited by 1,087
  2. 2024DriveVLM: The Convergence of Autonomous Driving and Large Vision-Language Modelscited by 490
  3. 2023NoMaD: Goal Masked Diffusion Policies for Navigation and Explorationcited by 248
  4. 2023Parting with Misconceptions about Learning-based Vehicle Motion Planningcited by 242
  5. 2024EMMA: End-to-End Multimodal Model for Autonomous Drivingcited by 231
  6. 2023MotionDiffuser: Controllable Multi-Agent Motion Prediction using Diffusioncited by 213
  7. 2024Understanding World or Predicting Future? A Comprehensive Survey of World Modelscited by 166
  8. 2023Waymax: An Accelerated, Data-Driven Simulator for Large-Scale Autonomous Driving Researchcited by 160
  9. 2023MotionLM: Multi-Agent Motion Forecasting as Language Modelingcited by 156
  10. 2023GameFormer: Game-theoretic Modeling and Learning of Transformer-based Interactive Prediction and Planning for Autonomous Drivingcited by 141
  11. 2025Diffusion-Based Planning for Autonomous Driving with Flexible Guidancecited by 141
  12. 2022Imitation Is Not Enough: Robustifying Imitation with Reinforcement Learning for Challenging Driving Scenarioscited by 129
38 more
  1. 2022ViP3D: End-to-end Visual Trajectory Prediction via 3D Agent Queriescited by 117
  2. 2023Language-Guided Traffic Simulation via Scene-Level Diffusioncited by 111
  3. 2024OpenEMMA: Open-Source Multimodal Model for End-to-End Autonomous Drivingcited by 93
  4. 2023The Waymo Open Sim Agents Challengecited by 86
  5. 2022GoRela: Go Relative for Viewpoint-Invariant Motion Forecastingcited by 68
  6. 2025WOD-E2E: Waymo Open Dataset for End-to-End Driving in Challenging Long-tail Scenarioscited by 62
  7. 2023PillarNeXt: Rethinking Network Designs for 3D Object Detection in LiDAR Point Cloudscited by 54
  8. 2024UniTraj: A Unified Framework for Scalable Vehicle Trajectory Predictioncited by 52
  9. 2024BehaviorGPT: Smart Agent Simulation for Autonomous Driving with Next-Patch Predictioncited by 50
  10. 2023ProphNet: Efficient Agent-Centric Motion Forecasting with Anchor-Informed Proposalscited by 48
  11. 2024SceneDiffuser: Efficient and Controllable Driving Simulation Initialization and Rolloutcited by 48
  12. 2023Scenario Diffusion: Controllable Driving Scenario Generation With Diffusioncited by 47
  13. 2023DTPP: Differentiable Joint Conditional Prediction and Cost Evaluation for Tree Policy Planning in Autonomous Drivingcited by 46
  14. 2023Traj-MAE: Masked Autoencoders for Trajectory Predictioncited by 42
  15. 2022JFP: Joint Future Prediction with Interactive Multi-Agent Modeling for Autonomous Drivingcited by 40
  16. 2024Solving Motion Planning Tasks with a Scalable Generative Modelcited by 40
  17. 2024HPNet: Dynamic Trajectory Forecasting with Historical Prediction Attentioncited by 38
  18. 2023Empowering Autonomous Driving with Large Language Models: A Safety Perspectivecited by 34
  19. 2024SmartRefine: A Scenario-Adaptive Refinement Framework for Efficient Motion Predictioncited by 34
  20. 2023Real-Time Motion Prediction via Heterogeneous Polyline Transformer with Relative Pose Encodingcited by 33
  21. 2024WcDT: World-centric Diffusion Transformer for Traffic Scene Generationcited by 32
  22. 2025MoFlow: One-Step Flow Matching for Human Trajectory Forecasting via Implicit Maximum Likelihood Estimation based Distillationcited by 31
  23. 2023The Integration of Prediction and Planning in Deep Learning Automated Driving Systems: A Reviewcited by 29
  24. 2024SURDS: Benchmarking Spatial Understanding and Reasoning in Driving Scenarios with Vision Language Modelscited by 29
  25. 2024DriveGPT: Scaling Autoregressive Behavior Models for Drivingcited by 29
  26. 2023WOMD-LiDAR: Raw Sensor Dataset Benchmark for Motion Forecastingcited by 28
  27. 2023SEPT: Towards Efficient Scene Representation Learning for Motion Predictioncited by 27
  28. 2024SIMPL: A Simple and Efficient Multi-agent Motion Prediction Baseline for Autonomous Drivingcited by 27
  29. 2024Can We Trust Embodied Agents? Exploring Backdoor Attacks against Embodied LLM-based Decision-Making Systemscited by 27
  30. 2024Improving Agent Behaviors with RL Fine-tuning for Autonomous Drivingcited by 27
  31. 2022GATraj: A Graph- and Attention-based Multi-Agent Trajectory Prediction Modelcited by 26
  32. 2023Multiverse Transformer: 1st Place Solution for Waymo Open Sim Agents Challenge 2023cited by 25
  33. 2024SMART: Scalable Multi-agent Real-time Motion Generation via Next-token Predictioncited by 25
  34. 2022R-Pred: Two-Stage Motion Prediction Via Tube-Query Attention-Based Trajectory Refinementcited by 24
  35. 2023Robots That Can See: Leveraging Human Pose for Trajectory Predictioncited by 23
  36. 2023Large Trajectory Models are Scalable Motion Predictors and Plannerscited by 23
  37. 2023Towards Motion Forecasting with Real-World Perception Inputs: Are End-to-End Approaches Competitive?cited by 22
  38. 2024Causal Composition Diffusion Model for Closed-loop Traffic Generationcited by 22

50 most cited of 280 All 280 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2021Multi-modal Motion Prediction with Transformer-based Neural Network for Autonomous Driving
  2. 2025HAMF: A Hybrid Attention-Mamba Framework for Joint Scene Context Understanding and Future Motion Representation Learning
  3. 2021Scene Transformer: A unified architecture for predicting multiple agent trajectories
  4. 2024DeMo: Decoupling Motion Forecasting into Directional Intentions and Dynamic States
  5. 2026Drive-HWM: Hierarchical World Models for Dynamic-Latent Guided Autonomous Driving
  6. 2022PreTR: Spatio-Temporal Non-Autoregressive Trajectory Prediction Transformer
  7. 2024Traj-LLM: A New Exploration for Empowering Trajectory Prediction with Pre-trained Large Language Models
  8. 2024Human Insights Driven Latent Space for Different Driving Perspectives: A Unified Encoder for Efficient Multi-Task Inference