# Trajectory Forecasting from Detection with Uncertainty-Aware Motion Encoding

arXiv: https://arxiv.org/abs/2202.01478 · HTML: https://mlsanity.com/paper/2202.01478/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 51 references that are in the map.
- First posted on arXiv 3 Feb 2022.

## Builds on (most cited first, 30 of 38)

- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [PointPillars: Fast Encoders for Object Detection from Point Clouds](https://mlsanity.com/paper/1812.05784/) (2018)
- [Center-based 3D Object Detection and Tracking](https://mlsanity.com/paper/2006.11275/) (2020)
- [DenseTNT: End-to-end Trajectory Prediction from Dense Goal Sets](https://mlsanity.com/paper/2108.09640/) (2021)
- [CoverNet: Multimodal Behavior Prediction using Trajectory Sets](https://mlsanity.com/paper/1911.10298/) (2019)
- [End-to-end Interpretable Neural Motion Planner](https://mlsanity.com/paper/2101.06679/) (2021)
- [It Is Not the Journey but the Destination: Endpoint Conditioned Trajectory Prediction](https://mlsanity.com/paper/2004.02025/) (2020)
- [Fast and Furious: Real Time End-to-End 3D Detection, Tracking and Motion Forecasting with a Single Convolutional Net](https://mlsanity.com/paper/2012.12395/) (2020)
- [Spatio-Temporal Graph Transformer Networks for Pedestrian Trajectory Prediction](https://mlsanity.com/paper/2005.08514/) (2020)
- [Multimodal Motion Prediction with Stacked Transformers](https://mlsanity.com/paper/2103.11624/) (2021)
- [IntentNet: Learning to Predict Intention from Raw Sensor Data](https://mlsanity.com/paper/2101.07907/) (2021)
- [PnPNet: End-to-End Perception and Prediction with Tracking in the Loop](https://mlsanity.com/paper/2005.14711/) (2020)
- [Multi-Agent Tensor Fusion for Contextual Trajectory Prediction](https://mlsanity.com/paper/1904.04776/) (2019)
- [TrafficPredict: Trajectory Prediction for Heterogeneous Traffic-Agents](https://mlsanity.com/paper/1811.02146/) (2018)
- [SR-LSTM: State Refinement for LSTM towards Pedestrian Trajectory Prediction](https://mlsanity.com/paper/1903.02793/) (2019)
- [Spatially-Aware Graph Neural Networks for Relational Behavior Forecasting from Sensor Data](https://mlsanity.com/paper/1910.08233/) (2019)
- [SGCN:Sparse Graph Convolution Network for Pedestrian Trajectory Prediction](https://mlsanity.com/paper/2104.01528/) (2021)
- [TPCN: Temporal Point Cloud Networks for Motion Forecasting](https://mlsanity.com/paper/2103.03067/) (2021)
- [TraPHic: Trajectory Prediction in Dense and Heterogeneous Traffic Using Weighted Interactions](https://mlsanity.com/paper/1812.04767/) (2018)
- [PiP: Planning-informed Trajectory Prediction for Autonomous Driving](https://mlsanity.com/paper/2003.11476/) (2020)
- [TPNet: Trajectory Proposal Network for Motion Prediction](https://mlsanity.com/paper/2004.12255/) (2020)
- [Tracking The Untrackable: Learning To Track Multiple Cues with Long-Term Dependencies](https://mlsanity.com/paper/1701.01909/) (2017)
- [SSN: Shape Signature Networks for Multi-class Object Detection from Point Clouds](https://mlsanity.com/paper/2004.02774/) (2020)
- [MANTRA: Memory Augmented Networks for Multiple Trajectory Prediction](https://mlsanity.com/paper/2006.03340/) (2020)
- [DSDNet: Deep Structured self-Driving Network](https://mlsanity.com/paper/2008.06041/) (2020)
- [Trajectory Prediction with Latent Belief Energy-Based Model](https://mlsanity.com/paper/2104.03086/) (2021)
- [Online Multi-Target Tracking Using Recurrent Neural Networks](https://mlsanity.com/paper/1604.03635/) (2016)
- [TITAN: Future Forecast using Action Priors](https://mlsanity.com/paper/2003.13886/) (2020)
- [Goal-GAN: Multimodal Trajectory Prediction Based on Goal Position Estimation](https://mlsanity.com/paper/2010.01114/) (2020)
- [Robust Multi-Modality Multi-Object Tracking](https://mlsanity.com/paper/1909.03850/) (2019)

## Built on by (most cited first, 8 of 8)

- [ViP3D: End-to-end Visual Trajectory Prediction via 3D Agent Queries](https://mlsanity.com/paper/2208.01582/) (2022)
- [UniTraj: A Unified Framework for Scalable Vehicle Trajectory Prediction](https://mlsanity.com/paper/2403.15098/) (2024)
- [Jointly Contrastive Representation Learning on Road Network and Trajectory](https://mlsanity.com/paper/2209.06389/) (2022)
- [Behavioral Intention Prediction in Driving Scenes: A Survey](https://mlsanity.com/paper/2211.00385/) (2022)
- [Towards Motion Forecasting with Real-World Perception Inputs: Are End-to-End Approaches Competitive?](https://mlsanity.com/paper/2306.09281/) (2023)
- [Motion Forecasting for Autonomous Vehicles: A Survey](https://mlsanity.com/paper/2502.08664/) (2025)
- [A Critical Perceptual Pre-trained Model for Complex Trajectory Recovery](https://mlsanity.com/paper/2311.02631/) (2023)
- [VisionTraj: A Noise-Robust Trajectory Recovery Framework based on Large-scale Camera Network](https://mlsanity.com/paper/2312.06428/) (2023)

## Abstract (arXiv)

Trajectory forecasting is critical for autonomous platforms to make safe planning and actions. Currently, most trajectory forecasting methods assume that object trajectories have been extracted and directly develop trajectory predictors based on the ground truth trajectories. However, this assumption does not hold in practical situations. Trajectories obtained from object detection and tracking are inevitably noisy, which could cause serious forecasting errors to predictors built on ground truth trajectories. In this paper, we propose a trajectory predictor directly based on detection results without relying on explicitly formed trajectories. Different from the traditional methods which encode the motion cue of an agent based on its clearly defined trajectory, we extract the motion information only based on the affinity cues among detection results, in which an affinity-aware state update mechanism is designed to take the uncertainty of association into account. In addition, considering that there could be multiple plausible matching candidates, we aggregate the states of them. This design relaxes the undesirable effect of noisy trajectory obtained from data association. Extensive ablation experiments validate the effectiveness of our method and its generalization ability on different detectors. Cross-comparison to other forecasting schemes further proves the superiority of our method. Code will be released upon acceptance.