# Interpretable Self-Aware Neural Networks for Robust Trajectory Prediction

arXiv: https://arxiv.org/abs/2211.08701 · HTML: https://mlsanity.com/paper/2211.08701/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 51 of its 60 references that are in the map.
- First posted on arXiv 16 Nov 2022.

## Builds on (most cited first, 30 of 51)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks](https://mlsanity.com/paper/1610.02136/) (2016)
- [What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?](https://mlsanity.com/paper/1703.04977/) (2017)
- [Variational Inference with Normalizing Flows](https://mlsanity.com/paper/1505.05770/) (2015)
- [Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift](https://mlsanity.com/paper/1906.02530/) (2019)
- [Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Images](https://mlsanity.com/paper/1412.1897/) (2014)
- [Evidential Deep Learning to Quantify Classification Uncertainty](https://mlsanity.com/paper/1806.01768/) (2018)
- [A Survey of Uncertainty in Deep Neural Networks](https://mlsanity.com/paper/2107.03342/) (2021)
- [Improving Variational Inference with Inverse Autoregressive Flow](https://mlsanity.com/paper/1606.04934/) (2016)
- [Predictive Uncertainty Estimation via Prior Networks](https://mlsanity.com/paper/1802.10501/) (2018)
- [Likelihood Ratios for Out-of-Distribution Detection](https://mlsanity.com/paper/1906.02845/) (2019)
- [MultiPath: Multiple Probabilistic Anchor Trajectory Hypotheses for Behavior Prediction](https://mlsanity.com/paper/1910.05449/) (2019)
- [Deep Evidential Regression](https://mlsanity.com/paper/1910.02600/) (2019)
- [TNT: Target-driveN Trajectory Prediction](https://mlsanity.com/paper/2008.08294/) (2020)
- [CoverNet: Multimodal Behavior Prediction using Trajectory Sets](https://mlsanity.com/paper/1911.10298/) (2019)
- [Stochastic Variational Video Prediction](https://mlsanity.com/paper/1710.11252/) (2017)
- [Human Motion Trajectory Prediction: A Survey](https://mlsanity.com/paper/1905.06113/) (2019)
- [Evaluating Scalable Bayesian Deep Learning Methods for Robust Computer Vision](https://mlsanity.com/paper/1906.01620/) (2019)
- [From Goals, Waypoints & Paths To Long Term Human Trajectory Forecasting](https://mlsanity.com/paper/2012.01526/) (2020)
- [Posterior Network: Uncertainty Estimation without OOD Samples via Density-Based Pseudo-Counts](https://mlsanity.com/paper/2006.09239/) (2020)
- [Can Autonomous Vehicles Identify, Recover From, and Adapt to Distribution Shifts?](https://mlsanity.com/paper/2006.14911/) (2020)
- [Out-of-Distribution Detection Using an Ensemble of Self Supervised Leave-out Classifiers](https://mlsanity.com/paper/1809.03576/) (2018)
- [Unsupervised Out-of-Distribution Detection by Maximum Classifier Discrepancy](https://mlsanity.com/paper/1908.04951/) (2019)
- [The Limits and Potentials of Deep Learning for Robotics](https://mlsanity.com/paper/1804.06557/) (2018)
- [An Uncertain Future: Forecasting from Static Images using Variational Autoencoders](https://mlsanity.com/paper/1606.07873/) (2016)

## Built on by (most cited first, 22 of 22)

- [Driving Everywhere with Large Language Model Policy Adaptation](https://mlsanity.com/paper/2402.05932/) (2024)
- [Can We Detect Failures Without Failure Data? Uncertainty-Aware Runtime Failure Detection for Imitation Learning Policies](https://mlsanity.com/paper/2503.08558/) (2025)
- [A Comprehensive Survey on Evidential Deep Learning and Its Applications](https://mlsanity.com/paper/2409.04720/) (2024)
- [SEAL: Towards Safe Autonomous Driving via Skill-Enabled Adversary Learning for Closed-Loop Scenario Generation](https://mlsanity.com/paper/2409.10320/) (2024)
- [Four Principles for Physically Interpretable World Models](https://mlsanity.com/paper/2503.02143/) (2025)
- [Rule-Based Error Detection and Correction to Operationalize Movement Trajectory Classification](https://mlsanity.com/paper/2308.14250/) (2023)
- [RedMotion: Motion Prediction via Redundancy Reduction](https://mlsanity.com/paper/2306.10840/) (2023)
- [Hyperdimensional Uncertainty Quantification for Multimodal Uncertainty Fusion in Autonomous Vehicles Perception](https://mlsanity.com/paper/2503.20011/) (2025)
- [Self-Aware Trajectory Prediction for Safe Autonomous Driving](https://mlsanity.com/paper/2305.09147/) (2023)
- [Stochasticity in Motion: An Information-Theoretic Approach to Trajectory Prediction](https://mlsanity.com/paper/2410.01628/) (2024)
- [TrajEvo: Designing Trajectory Prediction Heuristics via LLM-driven Evolution](https://mlsanity.com/paper/2505.04480/) (2025)
- [TrajEvo: Trajectory Prediction Heuristics Design via LLM-driven Evolution](https://mlsanity.com/paper/2508.05616/) (2025)
- [SceneMotion: From Agent-Centric Embeddings to Scene-Wide Forecasts](https://mlsanity.com/paper/2408.01537/) (2024)
- [LoRD: Adapting Differentiable Driving Policies to Distribution Shifts](https://mlsanity.com/paper/2410.09681/) (2024)
- [Physically Interpretable World Models via Weakly Supervised Representation Learning](https://mlsanity.com/paper/2412.12870/) (2024)
- [Uncertainty-Aware Trajectory Prediction via Rule-Regularized Heteroscedastic Deep Classification](https://mlsanity.com/paper/2504.13111/) (2025)
- [The Case for Negative Data: From Crash Reports to Counterfactuals for Reasonable Driving](https://mlsanity.com/paper/2509.18626/) (2025)
- [RuleFuser: An Evidential Bayes Approach for Rule Injection in Imitation Learned Planners and Predictors for Robustness under Distribution Shifts](https://mlsanity.com/paper/2405.11139/) (2024)
- [Geospatial Trajectory Generation via Efficient Abduction: Deployment for Independent Testing](https://mlsanity.com/paper/2407.06447/) (2024)
- [How Safe Will I Be Given What I Saw? Calibrated Prediction of Safety Chances for Image-Controlled Autonomy](https://mlsanity.com/paper/2508.09346/) (2025)
- [Generalized Evidential Deep Learning: From a Bayesian Perspective](https://mlsanity.com/paper/2605.25599/) (2026)
- [Evaluating Physical Consistency and Plausibility in Generative Scenario Models for Autonomous Driving](https://mlsanity.com/paper/2610.01581/) (2026)

## Abstract (arXiv)

Although neural networks have seen tremendous success as predictive models in a variety of domains, they can be overly confident in their predictions on out-of-distribution (OOD) data. To be viable for safety-critical applications, like autonomous vehicles, neural networks must accurately estimate their epistemic or model uncertainty, achieving a level of system self-awareness. Techniques for epistemic uncertainty quantification often require OOD data during training or multiple neural network forward passes during inference. These approaches may not be suitable for real-time performance on high-dimensional inputs. Furthermore, existing methods lack interpretability of the estimated uncertainty, which limits their usefulness both to engineers for further system development and to downstream modules in the autonomy stack. We propose the use of evidential deep learning to estimate the epistemic uncertainty over a low-dimensional, interpretable latent space in a trajectory prediction setting. We introduce an interpretable paradigm for trajectory prediction that distributes the uncertainty among the semantic concepts: past agent behavior, road structure, and social context. We validate our approach on real-world autonomous driving data, demonstrating superior performance over state-of-the-art baselines. Our code is available at: https://github.com/sisl/InterpretableSelfAwarePrediction.