# VideoGAN-based Trajectory Proposal for Automated Vehicles

arXiv: https://arxiv.org/abs/2506.16209 · HTML: https://mlsanity.com/paper/2506.16209/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 42 references that are in the map.
- First posted on arXiv 19 Jun 2025.

## Builds on (most cited first, 30 of 30)

- [Scalability in Perception for Autonomous Driving: Waymo Open Dataset](https://mlsanity.com/paper/1912.04838/) (2019)
- [Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networks](https://mlsanity.com/paper/1803.10892/) (2018)
- [A Survey of Deep Learning Techniques for Autonomous Driving](https://mlsanity.com/paper/1910.07738/) (2019)
- [ST-P3: End-to-end Vision-based Autonomous Driving via Spatial-Temporal Feature Learning](https://mlsanity.com/paper/2207.07601/) (2022)
- [Motion Transformer with Global Intention Localization and Local Movement Refinement](https://mlsanity.com/paper/2209.13508/) (2022)
- [FIERY: Future Instance Prediction in Bird's-Eye View from Surround Monocular Cameras](https://mlsanity.com/paper/2104.10490/) (2021)
- [MotionDiffuser: Controllable Multi-Agent Motion Prediction using Diffusion](https://mlsanity.com/paper/2306.03083/) (2023)
- [MP3: A Unified Model to Map, Perceive, Predict and Plan](https://mlsanity.com/paper/2101.06806/) (2021)
- [A Survey on Video Diffusion Models](https://mlsanity.com/paper/2310.10647/) (2023)
- [Stochastic Trajectory Prediction via Motion Indeterminacy Diffusion](https://mlsanity.com/paper/2203.13777/) (2022)
- [A Survey of Imitation Learning: Algorithms, Recent Developments, and Challenges](https://mlsanity.com/paper/2309.02473/) (2023)
- [HOME: Heatmap Output for future Motion Estimation](https://mlsanity.com/paper/2105.10968/) (2021)
- [Generating Long Videos of Dynamic Scenes](https://mlsanity.com/paper/2206.03429/) (2022)
- [HDGT: Heterogeneous Driving Graph Transformer for Multi-Agent Trajectory Prediction via Scene Encoding](https://mlsanity.com/paper/2205.09753/) (2022)
- [An Introduction to Deep Generative Modeling](https://mlsanity.com/paper/2103.05180/) (2021)
- [Towards learning-based planning:The nuPlan benchmark for real-world autonomous driving](https://mlsanity.com/paper/2403.04133/) (2024)
- [MotionNet: Joint Perception and Motion Prediction for Autonomous Driving Based on Bird's Eye View Maps](https://mlsanity.com/paper/2003.06754/) (2020)
- [LookOut: Diverse Multi-Future Prediction and Planning for Self-Driving](https://mlsanity.com/paper/2101.06547/) (2021)
- [TPNet: Trajectory Proposal Network for Motion Prediction](https://mlsanity.com/paper/2004.12255/) (2020)
- [Deep Imitative Models for Flexible Inference, Planning, and Control](https://mlsanity.com/paper/1810.06544/) (2018)
- [Occupancy Flow Fields for Motion Forecasting in Autonomous Driving](https://mlsanity.com/paper/2203.03875/) (2022)
- [Leveraging Future Relationship Reasoning for Vehicle Trajectory Prediction](https://mlsanity.com/paper/2305.14715/) (2023)
- [Implicit Occupancy Flow Fields for Perception and Prediction in Self-Driving](https://mlsanity.com/paper/2308.01471/) (2023)
- [The Integration of Prediction and Planning in Deep Learning Automated Driving Systems: A Review](https://mlsanity.com/paper/2308.05731/) (2023)
- [Scene Compliant Trajectory Forecast with Agent-Centric Spatio-Temporal Grids](https://mlsanity.com/paper/1909.07507/) (2019)
- [Machine Learning for Autonomous Vehicle's Trajectory Prediction: A comprehensive survey, Challenges, and Future Research Directions](https://mlsanity.com/paper/2307.07527/) (2023)
- [Generative Models in Decision Making: A Survey](https://mlsanity.com/paper/2502.17100/) (2025)
- [Vision-based Driver Assistance Systems: Survey, Taxonomy and Advances](https://mlsanity.com/paper/2104.12583/) (2021)
- [Kinematics-aware Trajectory Generation and Prediction with Latent Stochastic Differential Modeling](https://mlsanity.com/paper/2309.09317/) (2023)
- [Fine-Grained Behavior and Lane Constraints Guided Trajectory Prediction Method](https://mlsanity.com/paper/2503.21477/) (2025)

## Built on by (most cited first, 1 of 1)

- [Extended Field of View Analysis for VideoGAN-based Trajectory Generation](https://mlsanity.com/paper/2608.02289/) (2026)

## Abstract (arXiv)

Being able to generate realistic trajectory options is at the core of increasing the degree of automation of road vehicles. While model-driven, rule-based, and classical learning-based methods are widely used to tackle these tasks at present, they can struggle to effectively capture the complex, multimodal distributions of future trajectories. In this paper we investigate whether a generative adversarial network (GAN) trained on videos of bird's-eye view (BEV) traffic scenarios can generate statistically accurate trajectories that correctly capture spatial relationships between the agents. To this end, we propose a pipeline that uses low-resolution BEV occupancy grid videos as training data for a video generative model. From the generated videos of traffic scenarios we extract abstract trajectory data using single-frame object detection and frame-to-frame object matching. We particularly choose a GAN architecture for the fast training and inference times with respect to diffusion models. We obtain our best results within 100 GPU hours of training, with inference times under 20,ms. We demonstrate the physical realism of the proposed trajectories in terms of distribution alignment of spatial and dynamic parameters with respect to the ground truth videos from the Waymo Open Motion Dataset.