# FollowGen: A Scaled Noise Conditional Diffusion Model for Car-Following Trajectory Prediction

arXiv: https://arxiv.org/abs/2411.16747 · HTML: https://mlsanity.com/paper/2411.16747/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion and flow policies for robotics, one of 14 directions in Diffusion models.
- Builds on 28 of its 59 references that are in the map.
- First posted on arXiv 23 Nov 2024.

## Placed in

- Diffusion models / [Diffusion and flow policies for robotics](https://mlsanity.com/diffusion/diffusion-and-flow-policies-for-robotics/)

## Builds on (most cited first, 28 of 28)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Gate-Variants of Gated Recurrent Unit (GRU) Neural Networks](https://mlsanity.com/paper/1701.05923/) (2017)
- [MotionDiffuser: Controllable Multi-Agent Motion Prediction using Diffusion](https://mlsanity.com/paper/2306.03083/) (2023)
- [Stochastic Trajectory Prediction via Motion Indeterminacy Diffusion](https://mlsanity.com/paper/2203.13777/) (2022)
- [Multi-Agent Tensor Fusion for Contextual Trajectory Prediction](https://mlsanity.com/paper/1904.04776/) (2019)
- [An LSTM Network for Highway Trajectory Prediction](https://mlsanity.com/paper/1801.07962/) (2018)
- [Cross-Attention is All You Need: Adapting Pretrained Transformers for Machine Translation](https://mlsanity.com/paper/2104.08771/) (2021)
- [MUSE-VAE: Multi-Scale VAE for Environment-Aware Long Term Trajectory Prediction](https://mlsanity.com/paper/2201.07189/) (2022)
- [Graph-Based Spatial-Temporal Convolutional Network for Vehicle Trajectory Prediction in Autonomous Driving](https://mlsanity.com/paper/2109.12764/) (2021)
- [HAIM-DRL: Enhanced Human-in-the-loop Reinforcement Learning for Safe and Efficient Autonomous Driving](https://mlsanity.com/paper/2401.03160/) (2024)
- [Diff-RNTraj: A Structure-aware Diffusion Model for Road Network-constrained Trajectory Generation](https://mlsanity.com/paper/2402.07369/) (2024)
- [BAT: Behavior-Aware Human-Like Trajectory Prediction for Autonomous Driving](https://mlsanity.com/paper/2312.06371/) (2023)
- [GATraj: A Graph- and Attention-based Multi-Agent Trajectory Prediction Model](https://mlsanity.com/paper/2209.07857/) (2022)
- [End-to-End Trajectory Distribution Prediction Based on Occupancy Grid Maps](https://mlsanity.com/paper/2203.16910/) (2022)
- [S2TNet: Spatio-Temporal Transformer Networks for Trajectory Prediction in Autonomous Driving](https://mlsanity.com/paper/2206.10902/) (2022)
- [Variational Autoencoder-Based Vehicle Trajectory Prediction with an Interpretable Latent Space](https://mlsanity.com/paper/2103.13726/) (2021)
- [CRAT-Pred: Vehicle Trajectory Prediction with Crystal Graph Convolutional Neural Networks and Multi-Head Self-Attention](https://mlsanity.com/paper/2202.04488/) (2022)
- [EquiDiff: A Conditional Equivariant Diffusion Model For Trajectory Prediction](https://mlsanity.com/paper/2308.06564/) (2023)
- [Graph and Recurrent Neural Network-based Vehicle Trajectory Prediction For Highway Driving](https://mlsanity.com/paper/2107.03663/) (2021)
- [Interaction-Aware Trajectory Prediction of Connected Vehicles using CNN-LSTM Networks](https://mlsanity.com/paper/2005.12134/) (2020)
- [Diffusion-Based Environment-Aware Trajectory Prediction](https://mlsanity.com/paper/2403.11643/) (2024)
- [Graph-Based Interaction-Aware Multimodal 2D Vehicle Trajectory Prediction using Diffusion Graph Convolutional Networks](https://mlsanity.com/paper/2309.01981/) (2023)
- [TransFollower: Long-Sequence Car-Following Trajectory Prediction through Transformer](https://mlsanity.com/paper/2202.03183/) (2022)
- [Large Car-following Data Based on Lyft level-5 Open Dataset: Following Autonomous Vehicles vs. Human-driven Vehicles](https://mlsanity.com/paper/2305.18921/) (2023)
- [Traffic expertise meets residual RL: Knowledge-informed model-based residual reinforcement learning for CAV trajectory control](https://mlsanity.com/paper/2408.17380/) (2024)
- [MGF: Mixed Gaussian Flow for Diverse Trajectory Prediction](https://mlsanity.com/paper/2402.12238/) (2024)
- [A Physics Enhanced Residual Learning (PERL) Framework for Vehicle Trajectory Prediction](https://mlsanity.com/paper/2309.15284/) (2023)
- [Spatiotemporal Capsule Neural Network for Vehicle Trajectory Prediction](https://mlsanity.com/paper/2303.02880/) (2023)

## Built on by (most cited first, 3 of 3)

- [CurricuVLM: Towards Safe Autonomous Driving via Personalized Safety-Critical Curriculum Learning with Vision-Language Models](https://mlsanity.com/paper/2502.15119/) (2025)
- [Generative AI for Autonomous Driving: Frontiers and Opportunities](https://mlsanity.com/paper/2505.08854/) (2025)
- [HERMES: A Holistic End-to-End Risk-Aware Multimodal Embodied System with Vision-Language Models for Long-Tail Autonomous Driving](https://mlsanity.com/paper/2602.00993/) (2026)

## Abstract (arXiv)

Vehicle trajectory prediction is crucial for advancing autonomous driving and advanced driver assistance systems (ADAS). Although deep learning-based approaches - especially those utilizing transformer-based and generative models - have markedly improved prediction accuracy by capturing complex, non-linear patterns in vehicle dynamics and traffic interactions, they frequently overlook detailed car-following behaviors and the inter-vehicle interactions critical for real-world driving applications, particularly in fully autonomous or mixed traffic scenarios. To address the issue, this study introduces a scaled noise conditional diffusion model for car-following trajectory prediction, which integrates detailed inter-vehicular interactions and car-following dynamics into a generative framework, improving both the accuracy and plausibility of predicted trajectories. The model utilizes a novel pipeline to capture historical vehicle dynamics by scaling noise with encoded historical features within the diffusion process. Particularly, it employs a cross-attention-based transformer architecture to model intricate inter-vehicle dependencies, effectively guiding the denoising process and enhancing prediction accuracy. Experimental results on diverse real-world driving scenarios demonstrate the state-of-the-art performance and robustness of the proposed method.