Machine Learning for Autonomous Vehicle's Trajectory Prediction: A comprehensive survey, Challenges, and Future Research Directions
On the map: End-to-end autonomous driving → End-to-end system architecture
Abstract (arXiv)
Autonomous Vehicles (AVs) have emerged as a promising solution by replacing human drivers with advanced computer-aided decision-making systems. However, for AVs to effectively navigate the road, they must possess the capability to predict the future behavior of nearby traffic participants, similar to the predictive driving abilities of human drivers. Building upon existing literature is crucial to advance the field and develop a comprehensive understanding of trajectory prediction methods in the context of automated driving. To address this need, we have undertaken a comprehensive review that focuses on trajectory prediction methods for AVs, with a particular emphasis on machine learning techniques including deep learning and reinforcement learning-based approaches. We have extensively examined over two hundred studies related to trajectory prediction in the context of AVs. The paper begins with an introduction to the general problem of predicting vehicle trajectories and provides an overview of the key concepts and terminology used throughout. After providing a brief overview of conventional methods, this review conducts a comprehensive evaluation of several deep learning-based techniques. Each method is summarized briefly, accompanied by a detailed analysis of its strengths and weaknesses. The discussion further extends to reinforcement learning-based methods. This article also examines the various datasets and evaluation metrics that are commonly used in trajectory prediction tasks. Encouraging an unbiased and objective discussion, we compare two major learning processes, considering specific functional features. By identifying challenges in the existing literature and outlining potential research directions, this review significantly contributes to the advancement of knowledge in the domain of AV trajectory prediction.
Key facts. Cited by 23 papers in the map (data through 7 Oct 2026). Placed in End-to-end system architecture, one of 9 directions in End-to-end autonomous driving. Builds on 97 of its 246 references that are in the map. First posted on arXiv 12 Jul 2023.
Builds on
97 of 246 references.
References · 97
- 2017Attention Is All You Needcited by 59,932
- 2017Graph Attention Networkscited by 8,854
- 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
- 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
- 2014Sequence to Sequence Learning with Neural Networkscited by 2,974
- 2019A Comprehensive Survey on Graph Neural Networkscited by 2,906
- 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,338
- 2016Generative Adversarial Imitation Learningcited by 1,576
- 2018UNet++: A Nested U-Net Architecture for Medical Image Segmentationcited by 1,518
- 2018An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modelingcited by 1,447
- 2019Argoverse: 3D Tracking and Forecasting with Rich Mapscited by 728
- 2016Multi-View 3D Object Detection Network for Autonomous Drivingcited by 721
38 more
- 2023Argoverse 2: Next Generation Datasets for Self-Driving Perception and Forecastingcited by 701
- 2020Deep Reinforcement Learning for Autonomous Driving: A Surveycited by 698
- 2013Generating Sequences With Recurrent Neural Networkscited by 549
- 2021Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Datasetcited by 532
- 2020VectorNet: Encoding HD Maps and Agent Dynamics from Vectorized Representationcited by 528
- 2015Attention-Based Models for Speech Recognitioncited by 400
- 2018ChauffeurNet: Learning to Drive by Imitating the Best and Synthesizing the Worstcited by 391
- 2020Learning Lane Graph Representations for Motion Forecastingcited by 380
- 2020TNT: Target-driveN Trajectory Predictioncited by 345
- 2014Recurrent Models of Visual Attentioncited by 344
- 2021DenseTNT: End-to-end Trajectory Prediction from Dense Goal Setscited by 292
- 2018Multimodal Trajectory Predictions for Autonomous Driving using Deep Convolutional Networkscited by 289
- 2018The highD Dataset: A Drone Dataset of Naturalistic Vehicle Trajectories on German Highways for Validation of Highly Automated Driving Systemscited by 276
- 2019CoverNet: Multimodal Behavior Prediction using Trajectory Setscited by 259
- 2020One Thousand and One Hours: Self-driving Motion Prediction Datasetcited by 239
- 2019INTERACTION Dataset: An INTERnational, Adversarial and Cooperative moTION Dataset in Interactive Driving Scenarios with Semantic Mapscited by 231
- 2018Convolutional Social Pooling for Vehicle Trajectory Predictioncited by 221
- 2021Multimodal Motion Prediction with Stacked Transformerscited by 207
- 2018The ApolloScape Open Dataset for Autonomous Driving and its Applicationcited by 188
- 2019Multiple Futures Predictioncited by 178
- 2015Maximum Entropy Deep Inverse Reinforcement Learningcited by 173
- 2022Model-Based Imitation Learning for Urban Drivingcited by 149
- 2019Multi-Agent Tensor Fusion for Contextual Trajectory Predictioncited by 131
- 2021HOME: Heatmap Output for future Motion Estimationcited by 122
- 2021LaneRCNN: Distributed Representations for Graph-Centric Motion Forecastingcited by 112
- 2016Resnet in Resnet: Generalizing Residual Architecturescited by 110
- 2016DESPOT: Online POMDP Planning with Regularizationcited by 105
- 2021TPCN: Temporal Point Cloud Networks for Motion Forecastingcited by 94
- 2017Imitating Driver Behavior with Generative Adversarial Networkscited by 92
- 2018Uncertainty-aware Short-term Motion Prediction of Traffic Actors for Autonomous Drivingcited by 90
- 2019Multi-Head Attention for Multi-Modal Joint Vehicle Motion Forecastingcited by 87
- 2018TraPHic: Trajectory Prediction in Dense and Heterogeneous Traffic Using Weighted Interactionscited by 82
- 2018An LSTM Network for Highway Trajectory Predictioncited by 81
- 2018Multi-Modal Trajectory Prediction of Surrounding Vehicles with Maneuver based LSTMscited by 77
- 2019GRIP++: Enhanced Graph-based Interaction-aware Trajectory Prediction for Autonomous Drivingcited by 73
- 2021Multi-modal Motion Prediction with Transformer-based Neural Network for Autonomous Drivingcited by 73
- 2020Trajectory Forecasts in Unknown Environments Conditioned on Grid-Based Planscited by 72
- 2020MANTRA: Memory Augmented Networks for Multiple Trajectory Predictioncited by 69
50 most cited of 97 All 97 (JSON)
Built on by
Citing papers per year: 2023: 1; 2024: 5; 2025: 10; 2026: 7.
23 papers.
Sort:
By area:
- 2023Data-driven Traffic Simulation: A Comprehensive Reviewcited by 29
- 2025Trajectory Prediction for Autonomous Driving: Progress, Limitations, and Future Directionscited by 14
- 2024Reliable Trajectory Prediction and Uncertainty Quantification with Conditioned Diffusion Modelscited by 13
- 2024ComTraQ-MPC: Meta-Trained DQN-MPC Integration for Trajectory Tracking with Limited Active Localization Updatescited by 4
- 2025DOSE3 : Diffusion-based Out-of-distribution detection on SE(3) trajectoriescited by 3
- 2025Evaluating Generative Vehicle Trajectory Models for Traffic Intersection Dynamicscited by 3
- 2024Gradient-based Trajectory Optimization with Parallelized Differentiable Traffic Simulationcited by 2
- 2025The Components of Collaborative Joint Perception and Prediction -- A Conceptual Frameworkcited by 2
- 2025VideoGAN-based Trajectory Proposal for Automated Vehiclescited by 1
- 2025STRAP: Spatial-Temporal Risk-Attentive Vehicle Trajectory Prediction for Autonomous Drivingcited by 1
- 2025InterAct-Video: Reasoning-Rich Video QA for Urban Trafficcited by 1
- 2025To New Beginnings: A Survey of Unified Perception in Autonomous Vehicle Softwarecited by 1
11 more
- 2025X-TRACK: Physics-Aware xLSTM for Realistic Vehicle Trajectory Predictioncited by 1
- 2026From Observation to Prediction: LSTM for Vehicle Lane Change Forecasting on Highway On/Off-Rampscited by 1
- 2026RAPiD: Reward-Guided Consistency Distillation of Diffusion Planners for Real-Time Autonomous Drivingcited by 1
- 2024Towards Safe and Reliable Autonomous Driving: Dynamic Occupancy Set Predictioncited by 0
- 2024Maneuver Decision-Making with Trajectory Streams Prediction for Autonomous Vehiclescited by 0
- 2025Trajectory Prediction in Dynamic Object Tracking: A Critical Studycited by 0
- 2026Frozen LLMs as Map-Aware Spatio-Temporal Reasoners for Vehicle Trajectory Predictioncited by 0
- 2026From Cues to Horizons: Dynamic Risk Horizon Profiling for Trajectory Predictioncited by 0
- 2026LoRFT: Benchmarking Long-Range Vehicle Trajectory Reconstruction from Fixed Highway Camerascited by 0
- 2026Extended Field of View Analysis for VideoGAN-based Trajectory Generationcited by 0
- 2026Towards Collaborative Joint Perception and Prediction: Framework, Baseline Evaluation, and Deployment Perspectivescited by 0
- 2026Towards Collaborative Joint Perception and Prediction: Framework, Baseline Evaluation, and Deployment Perspectives10 Aug 2026
- 2026Extended Field of View Analysis for VideoGAN-based Trajectory Generation3 Aug 2026
- 2026LoRFT: Benchmarking Long-Range Vehicle Trajectory Reconstruction from Fixed Highway Cameras22 Jul 2026
- 2026From Cues to Horizons: Dynamic Risk Horizon Profiling for Trajectory Prediction30 May 2026
- 2026Frozen LLMs as Map-Aware Spatio-Temporal Reasoners for Vehicle Trajectory Prediction23 Apr 2026
- 2026RAPiD: Reward-Guided Consistency Distillation of Diffusion Planners for Real-Time Autonomous Driving7 Feb 2026
- 2026From Observation to Prediction: LSTM for Vehicle Lane Change Forecasting on Highway On/Off-Ramps21 Jan 2026
- 2025X-TRACK: Physics-Aware xLSTM for Realistic Vehicle Trajectory Prediction31 Oct 2025
- 2025To New Beginnings: A Survey of Unified Perception in Autonomous Vehicle Software28 Aug 2025
- 2025InterAct-Video: Reasoning-Rich Video QA for Urban Traffic19 Jul 2025
- 2025STRAP: Spatial-Temporal Risk-Attentive Vehicle Trajectory Prediction for Autonomous Driving11 Jul 2025
- 2025Trajectory Prediction in Dynamic Object Tracking: A Critical Study24 Jun 2025
11 more
- 2025VideoGAN-based Trajectory Proposal for Automated Vehicles19 Jun 2025
- 2025Evaluating Generative Vehicle Trajectory Models for Traffic Intersection Dynamics10 Jun 2025
- 2025Trajectory Prediction for Autonomous Driving: Progress, Limitations, and Future Directions5 Mar 2025
- 2025DOSE3 : Diffusion-based Out-of-distribution detection on SE(3) trajectories23 Feb 2025
- 2025The Components of Collaborative Joint Perception and Prediction -- A Conceptual Framework27 Jan 2025
- 2024Gradient-based Trajectory Optimization with Parallelized Differentiable Traffic Simulation21 Dec 2024
- 2024Maneuver Decision-Making with Trajectory Streams Prediction for Autonomous Vehicles16 Sep 2024
- 2024Reliable Trajectory Prediction and Uncertainty Quantification with Conditioned Diffusion Models23 May 2024
- 2024ComTraQ-MPC: Meta-Trained DQN-MPC Integration for Trajectory Tracking with Limited Active Localization Updates3 Mar 2024
- 2024Towards Safe and Reliable Autonomous Driving: Dynamic Occupancy Set Prediction29 Feb 2024
- 2023Data-driven Traffic Simulation: A Comprehensive Review24 Oct 2023
- 2023Data-driven Traffic Simulation: A Comprehensive Reviewcited by 29
- 2025Trajectory Prediction for Autonomous Driving: Progress, Limitations, and Future Directionscited by 14
- 2024ComTraQ-MPC: Meta-Trained DQN-MPC Integration for Trajectory Tracking with Limited Active Localization Updatescited by 4
- 2025DOSE3 : Diffusion-based Out-of-distribution detection on SE(3) trajectoriescited by 3
- 2025Evaluating Generative Vehicle Trajectory Models for Traffic Intersection Dynamicscited by 3
- 2024Gradient-based Trajectory Optimization with Parallelized Differentiable Traffic Simulationcited by 2
- 2025The Components of Collaborative Joint Perception and Prediction -- A Conceptual Frameworkcited by 2
- 2025VideoGAN-based Trajectory Proposal for Automated Vehiclescited by 1
- 2025STRAP: Spatial-Temporal Risk-Attentive Vehicle Trajectory Prediction for Autonomous Drivingcited by 1
- 2025InterAct-Video: Reasoning-Rich Video QA for Urban Trafficcited by 1
- 2025To New Beginnings: A Survey of Unified Perception in Autonomous Vehicle Softwarecited by 1
- 2025X-TRACK: Physics-Aware xLSTM for Realistic Vehicle Trajectory Predictioncited by 1
12 most cited of 19 All 19 (JSON)
- 2026Towards Collaborative Joint Perception and Prediction: Framework, Baseline Evaluation, and Deployment Perspectives10 Aug 2026
- 2026Extended Field of View Analysis for VideoGAN-based Trajectory Generation3 Aug 2026
- 2026LoRFT: Benchmarking Long-Range Vehicle Trajectory Reconstruction from Fixed Highway Cameras22 Jul 2026
- 2026From Cues to Horizons: Dynamic Risk Horizon Profiling for Trajectory Prediction30 May 2026
- 2026From Observation to Prediction: LSTM for Vehicle Lane Change Forecasting on Highway On/Off-Ramps21 Jan 2026
- 2025X-TRACK: Physics-Aware xLSTM for Realistic Vehicle Trajectory Prediction31 Oct 2025
- 2025To New Beginnings: A Survey of Unified Perception in Autonomous Vehicle Software28 Aug 2025
- 2025InterAct-Video: Reasoning-Rich Video QA for Urban Traffic19 Jul 2025
- 2025STRAP: Spatial-Temporal Risk-Attentive Vehicle Trajectory Prediction for Autonomous Driving11 Jul 2025
- 2025Trajectory Prediction in Dynamic Object Tracking: A Critical Study24 Jun 2025
- 2025VideoGAN-based Trajectory Proposal for Automated Vehicles19 Jun 2025
- 2025Evaluating Generative Vehicle Trajectory Models for Traffic Intersection Dynamics10 Jun 2025
12 newest of 19 All 19 (JSON)
Similar, not cited
Close in wording, no citation either way
- 2023End-to-end Autonomous Driving using Deep Learning: A Systematic Review
- 2019Review of Learning-based Longitudinal Motion Planning for Autonomous Vehicles: Research Gaps between Self-driving and Traffic Congestion
- 2024Towards learning-based planning:The nuPlan benchmark for real-world autonomous driving
- 2026Learning-Based Behavior Planning for Automated Driving: Real-World Integration and Deployment
- 2023Navigating Uncertainty: The Role of Short-Term Trajectory Prediction in Autonomous Vehicle Safety
- 2024Prediction Horizon Requirements for Automated Driving: Optimizing Safety, Comfort, and Efficiency
- 2023Recent Advancements in End-to-End Autonomous Driving using Deep Learning: A Survey
- 2024Autonomous Driving at Unsignalized Intersections: A Review of Decision-Making Challenges and Reinforcement Learning-Based Solutions