# Kinematics-aware Trajectory Generation and Prediction with Latent Stochastic Differential Modeling

arXiv: https://arxiv.org/abs/2309.09317 · HTML: https://mlsanity.com/paper/2309.09317/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 42 references that are in the map.
- First posted on arXiv 17 Sep 2023.

## Builds on (most cited first, 26 of 26)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Planning-oriented Autonomous Driving](https://mlsanity.com/paper/2212.10156/) (2022)
- [Argoverse: 3D Tracking and Forecasting with Rich Maps](https://mlsanity.com/paper/1911.02620/) (2019)
- [Learning Lane Graph Representations for Motion Forecasting](https://mlsanity.com/paper/2007.13732/) (2020)
- [TNT: Target-driveN Trajectory Prediction](https://mlsanity.com/paper/2008.08294/) (2020)
- [AgentFormer: Agent-Aware Transformers for Socio-Temporal Multi-Agent Forecasting](https://mlsanity.com/paper/2103.14023/) (2021)
- [Wayformer: Motion Forecasting via Simple & Efficient Attention Networks](https://mlsanity.com/paper/2207.05844/) (2022)
- [Scalable Gradients for Stochastic Differential Equations](https://mlsanity.com/paper/2001.01328/) (2020)
- [MotionDiffuser: Controllable Multi-Agent Motion Prediction using Diffusion](https://mlsanity.com/paper/2306.03083/) (2023)
- [Multimodal Motion Prediction with Stacked Transformers](https://mlsanity.com/paper/2103.11624/) (2021)
- [Generating Useful Accident-Prone Driving Scenarios via a Learned Traffic Prior](https://mlsanity.com/paper/2112.05077/) (2021)
- [A Survey on Safety-Critical Driving Scenario Generation -- A Methodological Perspective](https://mlsanity.com/paper/2202.02215/) (2022)
- [TPCN: Temporal Point Cloud Networks for Motion Forecasting](https://mlsanity.com/paper/2103.03067/) (2021)
- [GRIP++: Enhanced Graph-based Interaction-aware Trajectory Prediction for Autonomous Driving](https://mlsanity.com/paper/1907.07792/) (2019)
- [Efficient and Accurate Gradients for Neural SDEs](https://mlsanity.com/paper/2105.13493/) (2021)
- [Self-Driving like a Human driver instead of a Robocar: Personalized comfortable driving experience for autonomous vehicles](https://mlsanity.com/paper/2001.03908/) (2020)
- [CausalAF: Causal Autoregressive Flow for Safety-Critical Driving Scenario Generation](https://mlsanity.com/paper/2110.13939/) (2021)
- [CMTS: Conditional Multiple Trajectory Synthesizer for Generating Safety-critical Driving Scenarios](https://mlsanity.com/paper/1910.00099/) (2019)
- [Diverse Critical Interaction Generation for Planning and Planner Evaluation](https://mlsanity.com/paper/2103.00906/) (2021)
- [Semi-supervised Semantics-guided Adversarial Training for Trajectory Prediction](https://mlsanity.com/paper/2205.14230/) (2022)
- [TAE: A Semi-supervised Controllable Behavior-aware Trajectory Generator and Predictor](https://mlsanity.com/paper/2203.01261/) (2022)
- [Physics-Aware Safety-Assured Design of Hierarchical Neural Network based Planner](https://mlsanity.com/paper/2201.09140/) (2022)
- [Safety-driven Interactive Planning for Neural Network-based Lane Changing](https://mlsanity.com/paper/2201.09112/) (2022)
- [End-to-end Uncertainty-based Mitigation of Adversarial Attacks to Automated Lane Centering](https://mlsanity.com/paper/2103.00345/) (2021)
- [Safety-Assured Speculative Planning with Adaptive Prediction](https://mlsanity.com/paper/2307.11876/) (2023)
- [Statistical Characteristics of Driver Acceleration Behavior and Its Probability Model](https://mlsanity.com/paper/1907.01747/) (2019)

## Built on by (most cited first, 5 of 5)

- [Empowering Autonomous Driving with Large Language Models: A Safety Perspective](https://mlsanity.com/paper/2312.00812/) (2023)
- [Post-interactive Multimodal Trajectory Prediction for Autonomous Driving](https://mlsanity.com/paper/2503.09366/) (2025)
- [Belief-Based Offline Reinforcement Learning for Delay-Robust Policy Optimization](https://mlsanity.com/paper/2506.00131/) (2025)
- [VideoGAN-based Trajectory Proposal for Automated Vehicles](https://mlsanity.com/paper/2506.16209/) (2025)
- [Extended Field of View Analysis for VideoGAN-based Trajectory Generation](https://mlsanity.com/paper/2608.02289/) (2026)

## Abstract (arXiv)

Trajectory generation and trajectory prediction are two critical tasks in autonomous driving, which generate various trajectories for testing during development and predict the trajectories of surrounding vehicles during operation, respectively. In recent years, emerging data-driven deep learning-based methods have shown great promise for these two tasks in learning various traffic scenarios and improving average performance without assuming physical models. However, it remains a challenging problem for these methods to ensure that the generated/predicted trajectories are physically realistic. This challenge arises because learning-based approaches often function as opaque black boxes and do not adhere to physical laws. Conversely, existing model-based methods provide physically feasible results but are constrained by predefined model structures, limiting their capabilities to address complex scenarios. To address the limitations of these two types of approaches, we propose a new method that integrates kinematic knowledge into neural stochastic differential equations (SDE) and designs a variational autoencoder based on this latent kinematics-aware SDE (LK-SDE) to generate vehicle motions. Experimental results demonstrate that our method significantly outperforms both model-based and learning-based baselines in producing physically realistic and precisely controllable vehicle trajectories. Additionally, it performs well in predicting unobservable physical variables in the latent space.