# Uncertainty-Aware Trajectory Forecasting from Imperfect Tracking

arXiv: https://arxiv.org/abs/2608.30899 · HTML: https://mlsanity.com/paper/2608.30899/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 26 of its 47 references that are in the map.
- First posted on arXiv 31 Aug 2026.

## Builds on (most cited first, 26 of 26)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Knowledge Distillation: A Survey](https://mlsanity.com/paper/2006.05525/) (2020)
- [Simple Online and Realtime Tracking with a Deep Association Metric](https://mlsanity.com/paper/1703.07402/) (2017)
- [Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networks](https://mlsanity.com/paper/1803.10892/) (2018)
- [AgentFormer: Agent-Aware Transformers for Socio-Temporal Multi-Agent Forecasting](https://mlsanity.com/paper/2103.14023/) (2021)
- [Social-STGCNN: A Social Spatio-Temporal Graph Convolutional Neural Network for Human Trajectory Prediction](https://mlsanity.com/paper/2002.11927/) (2020)
- [BoT-SORT: Robust Associations Multi-Pedestrian Tracking](https://mlsanity.com/paper/2206.14651/) (2022)
- [Unifying distillation and privileged information](https://mlsanity.com/paper/1511.03643/) (2015)
- [StrongSORT: Make DeepSORT Great Again](https://mlsanity.com/paper/2202.13514/) (2022)
- [Stochastic Trajectory Prediction via Motion Indeterminacy Diffusion](https://mlsanity.com/paper/2203.13777/) (2022)
- [Human Trajectory Forecasting in Crowds: A Deep Learning Perspective](https://mlsanity.com/paper/2007.03639/) (2020)
- [EqMotion: Equivariant Multi-agent Motion Prediction with Invariant Interaction Reasoning](https://mlsanity.com/paper/2303.10876/) (2023)
- [Modality Distillation with Multiple Stream Networks for Action Recognition](https://mlsanity.com/paper/1806.07110/) (2018)
- [SocialVAE: Human Trajectory Prediction using Timewise Latents](https://mlsanity.com/paper/2203.08207/) (2022)
- [MUSE-VAE: Multi-Scale VAE for Environment-Aware Long Term Trajectory Prediction](https://mlsanity.com/paper/2201.07189/) (2022)
- [SingularTrajectory: Universal Trajectory Predictor Using Diffusion Model](https://mlsanity.com/paper/2403.18452/) (2024)
- [GATraj: A Graph- and Attention-based Multi-Agent Trajectory Prediction Model](https://mlsanity.com/paper/2209.07857/) (2022)
- [LG-Traj: LLM Guided Pedestrian Trajectory Prediction](https://mlsanity.com/paper/2403.08032/) (2024)
- [MART: MultiscAle Relational Transformer Networks for Multi-agent Trajectory Prediction](https://mlsanity.com/paper/2407.21635/) (2024)
- [MTP: Multi-Hypothesis Tracking and Prediction for Reduced Error Propagation](https://mlsanity.com/paper/2110.09481/) (2021)
- [OOSTraj: Out-of-Sight Trajectory Prediction With Vision-Positioning Denoising](https://mlsanity.com/paper/2404.02227/) (2024)
- [STFlow: Data-Coupled Flow Matching for Geometric Trajectory Simulation](https://mlsanity.com/paper/2505.18647/) (2025)
- [Towards Robust Human Trajectory Prediction in Raw Videos](https://mlsanity.com/paper/2108.08259/) (2021)
- [SceneAware: Scene-Constrained Pedestrian Trajectory Prediction with LLM-Guided Walkability](https://mlsanity.com/paper/2506.14144/) (2025)
- [VISTA: A Vision and Intent-Aware Social Attention Framework for Multi-Agent Trajectory Prediction](https://mlsanity.com/paper/2511.10203/) (2025)

## Abstract (arXiv)

Most trajectory forecasting models are trained on clean annotated histories, and are often evaluated under the same idealized assumption, although practical deployments rely on trajectories produced by imperfect multi-object trackers. The real-world observations exhibit localization jitter, missed or unstable detections, and data-association ambiguity, which are usually either ignored or removed through denoising. This paper instead treats tracking-derived reliability cues as an informative signal to be propagated to the predictor. We propose a plug-in uncertainty-aware formulation in which each observed state is encoded as an uncertain state representation, modeled by a Gaussian distribution whose covariance combines detection-level localization uncertainty and association-level ambiguity through the law of total variance. Existing backbones are adapted with minimal architectural changes: input trajectories are represented as Gaussian observations, and predicted trajectories are produced as Gaussian forecasts rather than deterministic coordinates. To train predictors that remain robust under structured observation noise, we combine temporally correlated Ornstein-Uhlenbeck perturbations with response-based knowledge distillation from a teacher trained on clean trajectories. Experiments on Oxford Town Centre and VIRAT using real tracker outputs, together with a complementary ETH/UCY pseudo-detection protocol, show that the proposed formulation improves displacement accuracy and the reliability-sharpness trade-off of probabilistic forecasts.