# RobustTP: End-to-End Trajectory Prediction for Heterogeneous Road-Agents in Dense Traffic with Noisy Sensor Inputs

arXiv: https://arxiv.org/abs/1907.08752 · HTML: https://mlsanity.com/paper/1907.08752/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Jul 2019.

## Built on by (most cited first, 16 of 16)

- [On Adversarial Robustness of Trajectory Prediction for Autonomous Vehicles](https://mlsanity.com/paper/2201.05057/) (2022)
- [CHAD: Charlotte Anomaly Dataset](https://mlsanity.com/paper/2212.09258/) (2022)
- [Pedestrian Trajectory Prediction in Pedestrian-Vehicle Mixed Environments: A Systematic Review](https://mlsanity.com/paper/2308.06419/) (2023)
- [Unlimited Neighborhood Interaction for Heterogeneous Trajectory Prediction](https://mlsanity.com/paper/2108.00238/) (2021)
- [METEOR:A Dense, Heterogeneous, and Unstructured Traffic Dataset With Rare Behaviors](https://mlsanity.com/paper/2109.07648/) (2021)
- [Learning the Pedestrian-Vehicle Interaction for Pedestrian Trajectory Prediction](https://mlsanity.com/paper/2202.05334/) (2022)
- [Deep Learning based Computer Vision Methods for Complex Traffic Environments Perception: A Review](https://mlsanity.com/paper/2211.05120/) (2022)
- [Multi-Agent Inverse Reinforcement Learning in Real World Unstructured Pedestrian Crowds](https://mlsanity.com/paper/2405.16439/) (2024)
- [iPLAN: Intent-Aware Planning in Heterogeneous Traffic via Distributed Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2306.06236/) (2023)
- [Scenario-based Evaluation of Prediction Models for Automated Vehicles](https://mlsanity.com/paper/2210.06553/) (2022)
- [VegaEdge: Edge AI Confluence Anomaly Detection for Real-Time Highway IoT-Applications](https://mlsanity.com/paper/2311.07880/) (2023)
- [Transfer-LMR: Heavy-Tail Driving Behavior Recognition in Diverse Traffic Scenarios](https://mlsanity.com/paper/2405.05354/) (2024)
- [Prompt-Driven Domain Adaptation for End-to-End Autonomous Driving via In-Context RL](https://mlsanity.com/paper/2511.12755/) (2025)
- [Collaborative-Distilled Diffusion Models (CDDM) for Accelerated and Lightweight Trajectory Prediction](https://mlsanity.com/paper/2510.00627/) (2025)
- [Are LLMs The Way Forward? A Case Study on LLM-Guided Reinforcement Learning for Decentralized Autonomous Driving](https://mlsanity.com/paper/2511.12751/) (2025)
- [Physics-Informed CNN-LSTM for Street-Scale Urban Flood Prediction: Reconciling Aggregate Accuracy and Street-Level Plausibility](https://mlsanity.com/paper/2607.25148/) (2026)

## Abstract (arXiv)

We present RobustTP, an end-to-end algorithm for predicting future trajectories of road-agents in dense traffic with noisy sensor input trajectories obtained from RGB cameras (either static or moving) through a tracking algorithm. In this case, we consider noise as the deviation from the ground truth trajectory. The amount of noise depends on the accuracy of the tracking algorithm. Our approach is designed for dense heterogeneous traffic, where the road agents corresponding to a mixture of buses, cars, scooters, bicycles, or pedestrians. RobustTP is an approach that first computes trajectories using a combination of a non-linear motion model and a deep learning-based instance segmentation algorithm. Next, these noisy trajectories are trained using an LSTM-CNN neural network architecture that models the interactions between road-agents in dense and heterogeneous traffic. Our trajectory prediction algorithm outperforms state-of-the-art methods for end-to-end trajectory prediction using sensor inputs. We achieve an improvement of upto 18% in average displacement error and an improvement ofup to 35.5% in final displacement error at the end of the prediction window (5 seconds) over the next best method. All experiments were set up on an Nvidia TiTan Xp GPU. Additionally, we release a software framework, TrackNPred. The framework consists of implementations of state-of-the-art tracking and trajectory prediction methods and tools to benchmark and evaluate them on real-world dense traffic datasets.