# Machine Learning for Autonomous Vehicle's Trajectory Prediction: A comprehensive survey, Challenges, and Future Research Directions

arXiv: https://arxiv.org/abs/2307.07527 · HTML: https://mlsanity.com/paper/2307.07527/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Placed in End-to-end system architecture, one of 9 directions in End-to-end autonomous driving.
- Builds on 97 of its 246 references that are in the map.
- First posted on arXiv 12 Jul 2023.

## Placed in

- End-to-end autonomous driving / [End-to-end system architecture](https://mlsanity.com/driving/end-to-end-system-architecture/)

## Builds on (most cited first, 30 of 97)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [A Comprehensive Survey on Graph Neural Networks](https://mlsanity.com/paper/1901.00596/) (2019)
- [Masked-attention Mask Transformer for Universal Image Segmentation](https://mlsanity.com/paper/2112.01527/) (2021)
- [Generative Adversarial Imitation Learning](https://mlsanity.com/paper/1606.03476/) (2016)
- [UNet++: A Nested U-Net Architecture for Medical Image Segmentation](https://mlsanity.com/paper/1807.10165/) (2018)
- [An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling](https://mlsanity.com/paper/1803.01271/) (2018)
- [Argoverse: 3D Tracking and Forecasting with Rich Maps](https://mlsanity.com/paper/1911.02620/) (2019)
- [Multi-View 3D Object Detection Network for Autonomous Driving](https://mlsanity.com/paper/1611.07759/) (2016)
- [Deep Reinforcement Learning for Autonomous Driving: A Survey](https://mlsanity.com/paper/2002.00444/) (2020)
- [Argoverse 2: Next Generation Datasets for Self-Driving Perception and Forecasting](https://mlsanity.com/paper/2301.00493/) (2023)
- [Generating Sequences With Recurrent Neural Networks](https://mlsanity.com/paper/1308.0850/) (2013)
- [Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Dataset](https://mlsanity.com/paper/2104.10133/) (2021)
- [VectorNet: Encoding HD Maps and Agent Dynamics from Vectorized Representation](https://mlsanity.com/paper/2005.04259/) (2020)
- [Attention-Based Models for Speech Recognition](https://mlsanity.com/paper/1506.07503/) (2015)
- [ChauffeurNet: Learning to Drive by Imitating the Best and Synthesizing the Worst](https://mlsanity.com/paper/1812.03079/) (2018)
- [Learning Lane Graph Representations for Motion Forecasting](https://mlsanity.com/paper/2007.13732/) (2020)
- [Recurrent Models of Visual Attention](https://mlsanity.com/paper/1406.6247/) (2014)
- [TNT: Target-driveN Trajectory Prediction](https://mlsanity.com/paper/2008.08294/) (2020)
- [DenseTNT: End-to-end Trajectory Prediction from Dense Goal Sets](https://mlsanity.com/paper/2108.09640/) (2021)
- [Multimodal Trajectory Predictions for Autonomous Driving using Deep Convolutional Networks](https://mlsanity.com/paper/1809.10732/) (2018)
- [The highD Dataset: A Drone Dataset of Naturalistic Vehicle Trajectories on German Highways for Validation of Highly Automated Driving Systems](https://mlsanity.com/paper/1810.05642/) (2018)
- [CoverNet: Multimodal Behavior Prediction using Trajectory Sets](https://mlsanity.com/paper/1911.10298/) (2019)
- [One Thousand and One Hours: Self-driving Motion Prediction Dataset](https://mlsanity.com/paper/2006.14480/) (2020)
- [INTERACTION Dataset: An INTERnational, Adversarial and Cooperative moTION Dataset in Interactive Driving Scenarios with Semantic Maps](https://mlsanity.com/paper/1910.03088/) (2019)
- [Convolutional Social Pooling for Vehicle Trajectory Prediction](https://mlsanity.com/paper/1805.06771/) (2018)
- [Multimodal Motion Prediction with Stacked Transformers](https://mlsanity.com/paper/2103.11624/) (2021)

## Built on by (most cited first, 23 of 23)

- [Data-driven Traffic Simulation: A Comprehensive Review](https://mlsanity.com/paper/2310.15975/) (2023)
- [Trajectory Prediction for Autonomous Driving: Progress, Limitations, and Future Directions](https://mlsanity.com/paper/2503.03262/) (2025)
- [Reliable Trajectory Prediction and Uncertainty Quantification with Conditioned Diffusion Models](https://mlsanity.com/paper/2405.14384/) (2024)
- [ComTraQ-MPC: Meta-Trained DQN-MPC Integration for Trajectory Tracking with Limited Active Localization Updates](https://mlsanity.com/paper/2403.01564/) (2024)
- [DOSE3 : Diffusion-based Out-of-distribution detection on SE(3) trajectories](https://mlsanity.com/paper/2502.16725/) (2025)
- [Evaluating Generative Vehicle Trajectory Models for Traffic Intersection Dynamics](https://mlsanity.com/paper/2506.08963/) (2025)
- [Gradient-based Trajectory Optimization with Parallelized Differentiable Traffic Simulation](https://mlsanity.com/paper/2412.16750/) (2024)
- [The Components of Collaborative Joint Perception and Prediction -- A Conceptual Framework](https://mlsanity.com/paper/2501.15860/) (2025)
- [STRAP: Spatial-Temporal Risk-Attentive Vehicle Trajectory Prediction for Autonomous Driving](https://mlsanity.com/paper/2507.08563/) (2025)
- [VideoGAN-based Trajectory Proposal for Automated Vehicles](https://mlsanity.com/paper/2506.16209/) (2025)
- [InterAct-Video: Reasoning-Rich Video QA for Urban Traffic](https://mlsanity.com/paper/2507.14743/) (2025)
- [To New Beginnings: A Survey of Unified Perception in Autonomous Vehicle Software](https://mlsanity.com/paper/2508.20892/) (2025)
- [X-TRACK: Physics-Aware xLSTM for Realistic Vehicle Trajectory Prediction](https://mlsanity.com/paper/2511.00266/) (2025)
- [From Observation to Prediction: LSTM for Vehicle Lane Change Forecasting on Highway On/Off-Ramps](https://mlsanity.com/paper/2601.14848/) (2026)
- [RAPiD: Reward-Guided Consistency Distillation of Diffusion Planners for Real-Time Autonomous Driving](https://mlsanity.com/paper/2602.07339/) (2026)
- [Towards Safe and Reliable Autonomous Driving: Dynamic Occupancy Set Prediction](https://mlsanity.com/paper/2402.19385/) (2024)
- [Maneuver Decision-Making with Trajectory Streams Prediction for Autonomous Vehicles](https://mlsanity.com/paper/2409.10165/) (2024)
- [Trajectory Prediction in Dynamic Object Tracking: A Critical Study](https://mlsanity.com/paper/2506.19341/) (2025)
- [Frozen LLMs as Map-Aware Spatio-Temporal Reasoners for Vehicle Trajectory Prediction](https://mlsanity.com/paper/2604.21479/) (2026)
- [From Cues to Horizons: Dynamic Risk Horizon Profiling for Trajectory Prediction](https://mlsanity.com/paper/2606.00857/) (2026)
- [LoRFT: Benchmarking Long-Range Vehicle Trajectory Reconstruction from Fixed Highway Cameras](https://mlsanity.com/paper/2607.19911/) (2026)
- [Extended Field of View Analysis for VideoGAN-based Trajectory Generation](https://mlsanity.com/paper/2608.02289/) (2026)
- [Towards Collaborative Joint Perception and Prediction: Framework, Baseline Evaluation, and Deployment Perspectives](https://mlsanity.com/paper/2608.09541/) (2026)

## Abstract (arXiv)

Autonomous Vehicles (AVs) have emerged as a promising solution by replacing human drivers with advanced computer-aided decision-making systems. However, for AVs to effectively navigate the road, they must possess the capability to predict the future behavior of nearby traffic participants, similar to the predictive driving abilities of human drivers. Building upon existing literature is crucial to advance the field and develop a comprehensive understanding of trajectory prediction methods in the context of automated driving. To address this need, we have undertaken a comprehensive review that focuses on trajectory prediction methods for AVs, with a particular emphasis on machine learning techniques including deep learning and reinforcement learning-based approaches. We have extensively examined over two hundred studies related to trajectory prediction in the context of AVs. The paper begins with an introduction to the general problem of predicting vehicle trajectories and provides an overview of the key concepts and terminology used throughout. After providing a brief overview of conventional methods, this review conducts a comprehensive evaluation of several deep learning-based techniques. Each method is summarized briefly, accompanied by a detailed analysis of its strengths and weaknesses. The discussion further extends to reinforcement learning-based methods. This article also examines the various datasets and evaluation metrics that are commonly used in trajectory prediction tasks. Encouraging an unbiased and objective discussion, we compare two major learning processes, considering specific functional features. By identifying challenges in the existing literature and outlining potential research directions, this review significantly contributes to the advancement of knowledge in the domain of AV trajectory prediction.