# Forecasting Trajectory and Behavior of Road-Agents Using Spectral Clustering in Graph-LSTMs

arXiv: https://arxiv.org/abs/1912.01118 · HTML: https://mlsanity.com/paper/1912.01118/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Dec 2019.

## Built on by (most cited first, 30 of 34)

- [On Adversarial Robustness of Trajectory Prediction for Autonomous Vehicles](https://mlsanity.com/paper/2201.05057/) (2022)
- [SUTD-TrafficQA: A Question Answering Benchmark and an Efficient Network for Video Reasoning over Traffic Events](https://mlsanity.com/paper/2103.15538/) (2021)
- [Leveraging Future Relationship Reasoning for Vehicle Trajectory Prediction](https://mlsanity.com/paper/2305.14715/) (2023)
- [Machine Learning for Autonomous Vehicle's Trajectory Prediction: A comprehensive survey, Challenges, and Future Research Directions](https://mlsanity.com/paper/2307.07527/) (2023)
- [Unlimited Neighborhood Interaction for Heterogeneous Trajectory Prediction](https://mlsanity.com/paper/2108.00238/) (2021)
- [Propagating State Uncertainty Through Trajectory Forecasting](https://mlsanity.com/paper/2110.03267/) (2021)
- [Trajectory Prediction for Autonomous Driving: Progress, Limitations, and Future Directions](https://mlsanity.com/paper/2503.03262/) (2025)
- [METEOR:A Dense, Heterogeneous, and Unstructured Traffic Dataset With Rare Behaviors](https://mlsanity.com/paper/2109.07648/) (2021)
- [D2-TPred: Discontinuous Dependency for Trajectory Prediction under Traffic Lights](https://mlsanity.com/paper/2207.10398/) (2022)
- [Leveraging Smooth Attention Prior for Multi-Agent Trajectory Prediction](https://mlsanity.com/paper/2203.04421/) (2022)
- [Imitation Learning of Hierarchical Driving Model: from Continuous Intention to Continuous Trajectory](https://mlsanity.com/paper/2010.10393/) (2020)
- [Learning the Pedestrian-Vehicle Interaction for Pedestrian Trajectory Prediction](https://mlsanity.com/paper/2202.05334/) (2022)
- [A Hybrid Rule-Based and Data-Driven Approach to Driver Modeling through Particle Filtering](https://mlsanity.com/paper/2108.12820/) (2021)
- [SOCIALMAPF: Optimal and Efficient Multi-Agent Path Finding with Strategic Agents for Social Navigation](https://mlsanity.com/paper/2210.08390/) (2022)
- [iPLAN: Intent-Aware Planning in Heterogeneous Traffic via Distributed Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2306.06236/) (2023)
- [Towards Safe Autonomy in Hybrid Traffic: Detecting Unpredictable Abnormal Behaviors of Human Drivers via Information Sharing](https://mlsanity.com/paper/2309.16716/) (2023)
- [StreamMOTP: Streaming and Unified Framework for Joint 3D Multi-Object Tracking and Trajectory Prediction](https://mlsanity.com/paper/2406.19844/) (2024)
- [Building Real-time Awareness of Out-of-distribution in Trajectory Prediction for Autonomous Vehicles](https://mlsanity.com/paper/2409.17277/) (2024)
- [Conditional Generative Adversarial Networks for Speed Control in Trajectory Simulation](https://mlsanity.com/paper/2103.11471/) (2021)
- [Decentralized Social Navigation with Non-Cooperative Robots via Bi-Level Optimization](https://mlsanity.com/paper/2306.08815/) (2023)
- [Exploring Map-based Features for Efficient Attention-based Vehicle Motion Prediction](https://mlsanity.com/paper/2205.13071/) (2022)
- [On Learning the Tail Quantiles of Driving Behavior Distributions via Quantile Regression and Flows](https://mlsanity.com/paper/2305.13106/) (2023)
- [Graph Neural Networks in Intelligent Transportation Systems: Advances, Applications and Trends](https://mlsanity.com/paper/2401.00713/) (2024)
- [Transfer-LMR: Heavy-Tail Driving Behavior Recognition in Diverse Traffic Scenarios](https://mlsanity.com/paper/2405.05354/) (2024)
- [Prompt-Driven Domain Adaptation for End-to-End Autonomous Driving via In-Context RL](https://mlsanity.com/paper/2511.12755/) (2025)
- [GPRAR: Graph Convolutional Network based Pose Reconstruction and Action Recognition for Human Trajectory Prediction](https://mlsanity.com/paper/2103.14113/) (2021)
- [A Socially Aware Reinforcement Learning Agent for The Single Track Road Problem](https://mlsanity.com/paper/2109.05486/) (2021)
- [Fighter flight trajectory prediction based on spatio-temporal graphcial attention network](https://mlsanity.com/paper/2405.08034/) (2024)
- [Reputation-Driven Asynchronous Federated Learning for Enhanced Trajectory Prediction with Blockchain](https://mlsanity.com/paper/2407.19428/) (2024)
- [Are LLMs The Way Forward? A Case Study on LLM-Guided Reinforcement Learning for Decentralized Autonomous Driving](https://mlsanity.com/paper/2511.12751/) (2025)

## Abstract (arXiv)

We present a novel approach for traffic forecasting in urban traffic scenarios using a combination of spectral graph analysis and deep learning. We predict both the low-level information (future trajectories) as well as the high-level information (road-agent behavior) from the extracted trajectory of each road-agent. Our formulation represents the proximity between the road agents using a weighted dynamic geometric graph (DGG). We use a two-stream graph-LSTM network to perform traffic forecasting using these weighted DGGs. The first stream predicts the spatial coordinates of road-agents, while the second stream predicts whether a road-agent is going to exhibit overspeeding, underspeeding, or neutral behavior by modeling spatial interactions between road-agents. Additionally, we propose a new regularization algorithm based on spectral clustering to reduce the error margin in long-term prediction (3-5 seconds) and improve the accuracy of the predicted trajectories. Moreover, we prove a theoretical upper bound on the regularized prediction error. We evaluate our approach on the Argoverse, Lyft, Apolloscape, and NGSIM datasets and highlight the benefits over prior trajectory prediction methods. In practice, our approach reduces the average prediction error by approximately 75% over prior algorithms and achieves a weighted average accuracy of 91.2% for behavior prediction. Additionally, our spectral regularization improves long-term prediction by up to 70%.