# Improving Movement Predictions of Traffic Actors in Bird's-Eye View Models using GANs and Differentiable Trajectory Rasterization

arXiv: https://arxiv.org/abs/2004.06247 · HTML: https://mlsanity.com/paper/2004.06247/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Apr 2020.

## Built on by (most cited first, 9 of 9)

- [Exploring Imitation Learning for Autonomous Driving with Feedback Synthesizer and Differentiable Rasterization](https://mlsanity.com/paper/2103.01882/) (2021)
- [T4P: Test-Time Training of Trajectory Prediction via Masked Autoencoder and Actor-specific Token Memory](https://mlsanity.com/paper/2403.10052/) (2024)
- [Generative Flows with Invertible Attentions](https://mlsanity.com/paper/2106.03959/) (2021)
- [Intention-aware Denoising Diffusion Model for Trajectory Prediction](https://mlsanity.com/paper/2403.09190/) (2024)
- [VT-Former: An Exploratory Study on Vehicle Trajectory Prediction for Highway Surveillance through Graph Isomorphism and Transformer](https://mlsanity.com/paper/2311.06623/) (2023)
- [Vision-based Multi-future Trajectory Prediction: A Survey](https://mlsanity.com/paper/2302.10463/) (2023)
- [A Gentle Introduction and Tutorial on Deep Generative Models in Transportation Research](https://mlsanity.com/paper/2410.07066/) (2024)
- [Informed Priors for Knowledge Integration in Trajectory Prediction](https://mlsanity.com/paper/2211.00348/) (2022)
- [The Prompt is Mightier than the Example](https://mlsanity.com/paper/2505.18485/) (2025)

## Abstract (arXiv)

One of the most critical pieces of the self-driving puzzle is the task of predicting future movement of surrounding traffic actors, which allows the autonomous vehicle to safely and effectively plan its future route in a complex world. Recently, a number of algorithms have been proposed to address this important problem, spurred by a growing interest of researchers from both industry and academia. Methods based on top-down scene rasterization on one side and Generative Adversarial Networks (GANs) on the other have shown to be particularly successful, obtaining state-of-the-art accuracies on the task of traffic movement prediction. In this paper we build upon these two directions and propose a raster-based conditional GAN architecture, powered by a novel differentiable rasterizer module at the input of the conditional discriminator that maps generated trajectories into the raster space in a differentiable manner. This simplifies the task for the discriminator as trajectories that are not scene-compliant are easier to discern, and allows the gradients to flow back forcing the generator to output better, more realistic trajectories. We evaluated the proposed method on a large-scale, real-world data set, showing that it outperforms state-of-the-art GAN-based baselines.