# Interpretable Deep Tracking

arXiv: https://arxiv.org/abs/2210.01266 · HTML: https://mlsanity.com/paper/2210.01266/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 14 of its 19 references that are in the map.
- First posted on arXiv 3 Oct 2022.

## Builds on (most cited first, 14 of 14)

- [PointPillars: Fast Encoders for Object Detection from Point Clouds](https://mlsanity.com/paper/1812.05784/) (2018)
- [VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection](https://mlsanity.com/paper/1711.06396/) (2017)
- [Center-based 3D Object Detection and Tracking](https://mlsanity.com/paper/2006.11275/) (2020)
- [End-to-end Interpretable Neural Motion Planner](https://mlsanity.com/paper/2101.06679/) (2021)
- [PnPNet: End-to-End Perception and Prediction with Tracking in the Loop](https://mlsanity.com/paper/2005.14711/) (2020)
- [GNN3DMOT: Graph Neural Network for 3D Multi-Object Tracking with Multi-Feature Learning](https://mlsanity.com/paper/2006.07327/) (2020)
- [Inducing Causal Structure for Interpretable Neural Networks](https://mlsanity.com/paper/2112.00826/) (2021)
- [Interpretable Learning for Self-Driving Cars by Visualizing Causal Attention](https://mlsanity.com/paper/1703.10631/) (2017)
- [Interpretable End-to-end Urban Autonomous Driving with Latent Deep Reinforcement Learning](https://mlsanity.com/paper/2001.08726/) (2020)
- [The Dangers of Post-hoc Interpretability: Unjustified Counterfactual Explanations](https://mlsanity.com/paper/1907.09294/) (2019)
- [Grounding Human-to-Vehicle Advice for Self-driving Vehicles](https://mlsanity.com/paper/1911.06978/) (2019)
- [Learnable Online Graph Representations for 3D Multi-Object Tracking](https://mlsanity.com/paper/2104.11747/) (2021)
- [MATS: An Interpretable Trajectory Forecasting Representation for Planning and Control](https://mlsanity.com/paper/2009.07517/) (2020)
- [Optimizing for Interpretability in Deep Neural Networks with Tree Regularization](https://mlsanity.com/paper/1908.05254/) (2019)

## Abstract (arXiv)

Imagine experiencing a crash as the passenger of an autonomous vehicle. Wouldn't you want to know why it happened? Current end-to-end optimizable deep neural networks (DNNs) in 3D detection, multi-object tracking, and motion forecasting provide little to no explanations about how they make their decisions. To help bridge this gap, we design an end-to-end optimizable multi-object tracking architecture and training protocol inspired by the recently proposed method of interchange intervention training (IIT). By enumerating different tracking decisions and associated reasoning procedures, we can train individual networks to reason about the possible decisions via IIT. Each network's decisions can be explained by the high-level structural causal model (SCM) it is trained in alignment with. Moreover, our proposed model learns to rank these outcomes, leveraging the promise of deep learning in end-to-end training, while being inherently interpretable.