# Trajectory-Prediction with Vision: A Survey

arXiv: https://arxiv.org/abs/2303.13354 · HTML: https://mlsanity.com/paper/2303.13354/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 28 references that are in the map.
- First posted on arXiv 15 Mar 2023.

## Builds on (most cited first, 19 of 19)

- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [Scalability in Perception for Autonomous Driving: Waymo Open Dataset](https://mlsanity.com/paper/1912.04838/) (2019)
- [Argoverse: 3D Tracking and Forecasting with Rich Maps](https://mlsanity.com/paper/1911.02620/) (2019)
- [AgentFormer: Agent-Aware Transformers for Socio-Temporal Multi-Agent Forecasting](https://mlsanity.com/paper/2103.14023/) (2021)
- [V2VNet: Vehicle-to-Vehicle Communication for Joint Perception and Prediction](https://mlsanity.com/paper/2008.07519/) (2020)
- [One Thousand and One Hours: Self-driving Motion Prediction Dataset](https://mlsanity.com/paper/2006.14480/) (2020)
- [Fast and Furious: Real Time End-to-End 3D Detection, Tracking and Motion Forecasting with a Single Convolutional Net](https://mlsanity.com/paper/2012.12395/) (2020)
- [Social Attention: Modeling Attention in Human Crowds](https://mlsanity.com/paper/1710.04689/) (2017)
- [PRECOG: PREdiction Conditioned On Goals in Visual Multi-Agent Settings](https://mlsanity.com/paper/1905.01296/) (2019)
- [PnPNet: End-to-End Perception and Prediction with Tracking in the Loop](https://mlsanity.com/paper/2005.14711/) (2020)
- [An LSTM Network for Highway Trajectory Prediction](https://mlsanity.com/paper/1801.07962/) (2018)
- [GRIP++: Enhanced Graph-based Interaction-aware Trajectory Prediction for Autonomous Driving](https://mlsanity.com/paper/1907.07792/) (2019)
- [LOKI: Long Term and Key Intentions for Trajectory Prediction](https://mlsanity.com/paper/2108.08236/) (2021)
- [MVFuseNet: Improving End-to-End Object Detection and Motion Forecasting through Multi-View Fusion of LiDAR Data](https://mlsanity.com/paper/2104.10772/) (2021)
- [A Random Finite Set Approach for Dynamic Occupancy Grid Maps with Real-Time Application](https://mlsanity.com/paper/1605.02406/) (2016)
- [Vision-RADAR fusion for Robotics BEV Detections: A Survey](https://mlsanity.com/paper/2302.06643/) (2023)
- [Surround-View Vision-based 3D Detection for Autonomous Driving: A Survey](https://mlsanity.com/paper/2302.06650/) (2023)
- [Transformer-Based Sensor Fusion for Autonomous Driving: A Survey](https://mlsanity.com/paper/2302.11481/) (2023)
- [3M3D: Multi-view, Multi-path, Multi-representation for 3D Object Detection](https://mlsanity.com/paper/2302.08231/) (2023)

## Built on by (most cited first, 4 of 4)

- [An Overview about Emerging Technologies of Autonomous Driving](https://mlsanity.com/paper/2306.13302/) (2023)
- [From Observation to Prediction: LSTM for Vehicle Lane Change Forecasting on Highway On/Off-Ramps](https://mlsanity.com/paper/2601.14848/) (2026)
- [Agent-Centric Social Trajectory Prediction: A Free Energy Principle Perspective](https://mlsanity.com/paper/2605.25748/) (2026)
- [SD-RouteFusion: Ego-Trajectory Prediction with SD-Map Route Conditioning](https://mlsanity.com/paper/2607.01139/) (2026)

## Abstract (arXiv)

To plan a safe and efficient route, an autonomous vehicle should anticipate future trajectories of other agents around it. Trajectory prediction is an extremely challenging task which recently gained a lot of attention in the autonomous vehicle research community. Trajectory-prediction forecasts future state of all the dynamic agents in the scene given their current and past states. A good prediction model can prevent collisions on the road, and hence the ultimate goal for autonomous vehicles: Collision rate: collisions per Million miles. The objective of this paper is to provide an overview of the field trajectory-prediction. We categorize the relevant algorithms into different classes so that researchers can follow through the trends in the trajectory-prediction research field. Moreover we also touch upon the background knowledge required to formulate a trajectory-prediction problem.