# You Mostly Walk Alone: Analyzing Feature Attribution in Trajectory Prediction

arXiv: https://arxiv.org/abs/2110.05304 · HTML: https://mlsanity.com/paper/2110.05304/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 45 references that are in the map.
- First posted on arXiv 11 Oct 2021.

## Builds on (most cited first, 22 of 22)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Visualizing and Understanding Convolutional Networks](https://mlsanity.com/paper/1311.2901/) (2013)
- [Learning Important Features Through Propagating Activation Differences](https://mlsanity.com/paper/1704.02685/) (2017)
- [Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networks](https://mlsanity.com/paper/1803.10892/) (2018)
- [The many Shapley values for model explanation](https://mlsanity.com/paper/1908.08474/) (2019)
- [Neural Relational Inference for Interacting Systems](https://mlsanity.com/paper/1802.04687/) (2018)
- [Social-STGCNN: A Social Spatio-Temporal Graph Convolutional Neural Network for Human Trajectory Prediction](https://mlsanity.com/paper/2002.11927/) (2020)
- [DESIRE: Distant Future Prediction in Dynamic Scenes with Interacting Agents](https://mlsanity.com/paper/1704.04394/) (2017)
- [Real Time Image Saliency for Black Box Classifiers](https://mlsanity.com/paper/1705.07857/) (2017)
- [It Is Not the Journey but the Destination: Endpoint Conditioned Trajectory Prediction](https://mlsanity.com/paper/2004.02025/) (2020)
- [Spatio-Temporal Graph Transformer Networks for Pedestrian Trajectory Prediction](https://mlsanity.com/paper/2005.08514/) (2020)
- [Feature relevance quantification in explainable AI: A causal problem](https://mlsanity.com/paper/1910.13413/) (2019)
- [Explaining Deep Neural Networks with a Polynomial Time Algorithm for Shapley Values Approximation](https://mlsanity.com/paper/1903.10992/) (2019)
- [SR-LSTM: State Refinement for LSTM towards Pedestrian Trajectory Prediction](https://mlsanity.com/paper/1903.02793/) (2019)
- [Amortized Causal Discovery: Learning to Infer Causal Graphs from Time-Series Data](https://mlsanity.com/paper/2006.10833/) (2020)
- [Overcoming Limitations of Mixture Density Networks: A Sampling and Fitting Framework for Multimodal Future Prediction](https://mlsanity.com/paper/1906.03631/) (2019)
- [On Exposing the Challenging Long Tail in Future Prediction of Traffic Actors](https://mlsanity.com/paper/2103.12474/) (2021)
- [Human Trajectory Prediction via Counterfactual Analysis](https://mlsanity.com/paper/2107.14202/) (2021)

## Built on by (most cited first, 23 of 23)

- [Stochastic Trajectory Prediction via Motion Indeterminacy Diffusion](https://mlsanity.com/paper/2203.13777/) (2022)
- [SocialVAE: Human Trajectory Prediction using Timewise Latents](https://mlsanity.com/paper/2203.08207/) (2022)
- [Towards Robust and Adaptive Motion Forecasting: A Causal Representation Perspective](https://mlsanity.com/paper/2111.14820/) (2021)
- [MacFormer: Map-Agent Coupled Transformer for Real-time and Robust Trajectory Prediction](https://mlsanity.com/paper/2308.10280/) (2023)
- [Identifiable Causal Representation Learning: Unsupervised, Multi-View, and Multi-Environment](https://mlsanity.com/paper/2406.13371/) (2024)
- [Interventional Behavior Prediction: Avoiding Overly Confident Anticipation in Interactive Prediction](https://mlsanity.com/paper/2204.08665/) (2022)
- [Resonance: Learning to Predict Social-Aware Pedestrian Trajectories as Co-Vibrations](https://mlsanity.com/paper/2412.02447/) (2024)
- [Sim-to-Real Causal Transfer: A Metric Learning Approach to Causally-Aware Interaction Representations](https://mlsanity.com/paper/2312.04540/) (2023)
- [LoRD: Adapting Differentiable Driving Policies to Distribution Shifts](https://mlsanity.com/paper/2410.09681/) (2024)
- [How Does Traffic Environment Quantitatively Affect the Autonomous Driving Prediction?](https://mlsanity.com/paper/2301.04414/) (2023)
- [Causal-discovery-based root-cause analysis and its application in time-series prediction error diagnosis](https://mlsanity.com/paper/2411.06990/) (2024)
- [ShuttleSHAP: A Turn-Based Feature Attribution Approach for Analyzing Forecasting Models in Badminton](https://mlsanity.com/paper/2312.10942/) (2023)
- [Informed Spectral Normalized Gaussian Processes for Trajectory Prediction](https://mlsanity.com/paper/2403.11966/) (2024)
- [Scene-Aware Explainable Multimodal Trajectory Prediction](https://mlsanity.com/paper/2410.16795/) (2024)
- [ITPNet: Towards Instantaneous Trajectory Prediction for Autonomous Driving](https://mlsanity.com/paper/2412.07369/) (2024)
- [INTENT: Trajectory Prediction Framework with Intention-Guided Contrastive Clustering](https://mlsanity.com/paper/2503.04952/) (2025)
- [Recent Advances in Multi-Agent Human Trajectory Prediction: A Comprehensive Review](https://mlsanity.com/paper/2506.14831/) (2025)
- [VIRTUE: Visual-Interactive Text-Image Universal Embedder](https://mlsanity.com/paper/2510.00523/) (2025)
- [PMM-Net: Single-stage Multi-agent Trajectory Prediction with Patching-based Embedding and Explicit Modal Modulation](https://mlsanity.com/paper/2410.19544/) (2024)
- [Hacking Predictors Means Hacking Cars: Using Sensitivity Analysis to Identify Trajectory Prediction Vulnerabilities for Autonomous Driving Security](https://mlsanity.com/paper/2401.10313/) (2024)
- [Super Agents and Confounders: Influence of surrounding agents on vehicle trajectory prediction](https://mlsanity.com/paper/2604.03463/) (2026)
- [RankFormer: A Propose-then-Select Transformer for Multi-Agent Multimodal Trajectory Prediction](https://mlsanity.com/paper/2604.07126/) (2026)
- [TSCA-Net: Temporal-Spatial Clique Attention for Interpretable Multimodal Pedestrian Trajectory Prediction](https://mlsanity.com/paper/2607.11939/) (2026)

## Abstract (arXiv)

Predicting the future trajectory of a moving agent can be easy when the past trajectory continues smoothly but is challenging when complex interactions with other agents are involved. Recent deep learning approaches for trajectory prediction show promising performance and partially attribute this to successful reasoning about agent-agent interactions. However, it remains unclear which features such black-box models actually learn to use for making predictions. This paper proposes a procedure that quantifies the contributions of different cues to model performance based on a variant of Shapley values. Applying this procedure to state-of-the-art trajectory prediction methods on standard benchmark datasets shows that they are, in fact, unable to reason about interactions. Instead, the past trajectory of the target is the only feature used for predicting its future. For a task with richer social interaction patterns, on the other hand, the tested models do pick up such interactions to a certain extent, as quantified by our feature attribution method. We discuss the limits of the proposed method and its links to causality