# Real Time Trajectory Prediction Using Deep Conditional Generative Models

arXiv: https://arxiv.org/abs/1909.03895 · HTML: https://mlsanity.com/paper/1909.03895/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Sep 2019.

## Built on by (most cited first, 11 of 11)

- [MotionDiffuser: Controllable Multi-Agent Motion Prediction using Diffusion](https://mlsanity.com/paper/2306.03083/) (2023)
- [Revisiting Energy Based Models as Policies: Ranking Noise Contrastive Estimation and Interpolating Energy Models](https://mlsanity.com/paper/2309.05803/) (2023)
- [Black-Box vs. Gray-Box: A Case Study on Learning Table Tennis Ball Trajectory Prediction with Spin and Impacts](https://mlsanity.com/paper/2305.15189/) (2023)
- [Multi-Camera Asynchronous Ball Localization and Trajectory Prediction with Factor Graphs and Human Poses](https://mlsanity.com/paper/2401.17185/) (2024)
- [Memory-Constrained Semantic Segmentation for Ultra-High Resolution UAV Imagery](https://mlsanity.com/paper/2310.04721/) (2023)
- [Egocentric Event-Based Vision for Ping Pong Ball Trajectory Prediction](https://mlsanity.com/paper/2506.07860/) (2025)
- [BilliardPhys-Bench: Benchmarking Physical Reasoning and Visual Dynamics of Multimodal LLMs](https://mlsanity.com/paper/2605.30900/) (2026)
- [Catching Spinning Table Tennis Balls in Simulation with End-to-End Curriculum Reinforcement Learning](https://mlsanity.com/paper/2503.01251/) (2025)
- [A prior information informed learning architecture for flying trajectory prediction](https://mlsanity.com/paper/2603.06863/) (2026)
- [1000 Rallies: An Event-Camera Dataset and Real-Time Learned Ball-State Estimation for Robotic Table Tennis](https://mlsanity.com/paper/2606.25620/) (2026)
- [Off the Rails: Hijacking the Scoring Head in Generative End-to-End Driving Planners with Safety-Violating Adversarial Perturbations](https://mlsanity.com/paper/2606.30807/) (2026)

## Abstract (arXiv)

Data driven methods for time series forecasting that quantify uncertainty open new important possibilities for robot tasks with hard real time constraints, allowing the robot system to make decisions that trade off between reaction time and accuracy in the predictions. Despite the recent advances in deep learning, it is still challenging to make long term accurate predictions with the low latency required by real time robotic systems. In this paper, we propose a deep conditional generative model for trajectory prediction that is learned from a data set of collected trajectories. Our method uses encoder and decoder deep networks that maps complete or partial trajectories to a Gaussian distributed latent space and back, allowing for fast inference of the future values of a trajectory given previous observations. The encoder and decoder networks are trained using stochastic gradient variational Bayes. In the experiments, we show that our model provides more accurate long term predictions with a lower latency that popular models for trajectory forecasting like recurrent neural networks or physical models based on differential equations. Finally, we test our proposed approach in a robot table tennis scenario to evaluate the performance of the proposed method in a robotic task with hard real time constraints.