# A Certifiable Security Patch for Object Tracking in Self-Driving Systems via Historical Deviation Modeling

arXiv: https://arxiv.org/abs/2207.08556 · HTML: https://mlsanity.com/paper/2207.08556/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 61 references that are in the map.
- First posted on arXiv 18 Jul 2022.

## Builds on (most cited first, 30 of 30)

- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015)
- [Towards Evaluating the Robustness of Neural Networks](https://mlsanity.com/paper/1608.04644/) (2016)
- [Adversarial examples in the physical world](https://mlsanity.com/paper/1607.02533/) (2016)
- [VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection](https://mlsanity.com/paper/1711.06396/) (2017)
- [PointRCNN: 3D Object Proposal Generation and Detection from Point Cloud](https://mlsanity.com/paper/1812.04244/) (2018)
- [Adversarial Examples for Semantic Segmentation and Object Detection](https://mlsanity.com/paper/1703.08603/) (2017)
- [Deep Continuous Fusion for Multi-Sensor 3D Object Detection](https://mlsanity.com/paper/2012.10992/) (2020)
- [A Tail-Index Analysis of Stochastic Gradient Noise in Deep Neural Networks](https://mlsanity.com/paper/1901.06053/) (2019)
- [Multiple Object Tracking: A Literature Review](https://mlsanity.com/paper/1409.7618/) (2014)
- [Physical Adversarial Examples for Object Detectors](https://mlsanity.com/paper/1807.07769/) (2018)
- [ShapeShifter: Robust Physical Adversarial Attack on Faster R-CNN Object Detector](https://mlsanity.com/paper/1804.05810/) (2018)
- [Self-Driving Cars: A Survey](https://mlsanity.com/paper/1901.04407/) (2019)
- [Physically Realizable Adversarial Examples for LiDAR Object Detection](https://mlsanity.com/paper/2004.00543/) (2020)
- [Fast Point R-CNN](https://mlsanity.com/paper/1908.02990/) (2019)
- [Invisible for both Camera and LiDAR: Security of Multi-Sensor Fusion based Perception in Autonomous Driving Under Physical-World Attacks](https://mlsanity.com/paper/2106.09249/) (2021)
- [Towards Robust LiDAR-based Perception in Autonomous Driving: General Black-box Adversarial Sensor Attack and Countermeasures](https://mlsanity.com/paper/2006.16974/) (2020)
- [Online Multi-Object Tracking with Dual Matching Attention Networks](https://mlsanity.com/paper/1902.00749/) (2019)
- [Dirty Road Can Attack: Security of Deep Learning based Automated Lane Centering under Physical-World Attack](https://mlsanity.com/paper/2009.06701/) (2020)
- [Adversarial Objects Against LiDAR-Based Autonomous Driving Systems](https://mlsanity.com/paper/1907.05418/) (2019)
- [AB3DMOT: A Baseline for 3D Multi-Object Tracking and New Evaluation Metrics](https://mlsanity.com/paper/2008.08063/) (2020)
- [DetectorGuard: Provably Securing Object Detectors against Localized Patch Hiding Attacks](https://mlsanity.com/paper/2102.02956/) (2021)
- [Drift with Devil: Security of Multi-Sensor Fusion based Localization in High-Level Autonomous Driving under GPS Spoofing (Extended Version)](https://mlsanity.com/paper/2006.10318/) (2020)
- [SoK: On the Semantic AI Security in Autonomous Driving](https://mlsanity.com/paper/2203.05314/) (2022)
- [Multi-Object Tracking with Multiple Cues and Switcher-Aware Classification](https://mlsanity.com/paper/1901.06129/) (2019)
- [Too Afraid to Drive: Systematic Discovery of Semantic DoS Vulnerability in Autonomous Driving Planning under Physical-World Attacks](https://mlsanity.com/paper/2201.04610/) (2022)
- [Data Driven Prediction Architecture for Autonomous Driving and its Application on Apollo Platform](https://mlsanity.com/paper/2006.06715/) (2020)
- [Who is in Control? Practical Physical Layer Attack and Defense for mmWave based Sensing in Autonomous Vehicles](https://mlsanity.com/paper/2011.10947/) (2020)
- [Multi-Object Tracking using Poisson Multi-Bernoulli Mixture Filtering for Autonomous Vehicles](https://mlsanity.com/paper/2103.07783/) (2021)
- [Real-Time Multiple Object Tracking - A Study on the Importance of Speed](https://mlsanity.com/paper/1709.03572/) (2017)

## Built on by (most cited first, 1 of 1)

- [3D Multiple Object Tracking on Autonomous Driving: A Literature Review](https://mlsanity.com/paper/2309.15411/) (2023)

## Abstract (arXiv)

Self-driving cars (SDC) commonly implement the perception pipeline to detect the surrounding obstacles and track their moving trajectories, which lays the ground for the subsequent driving decision making process. Although the security of obstacle detection in SDC is intensively studied, not until very recently the attackers start to exploit the vulnerability of the tracking module. Compared with solely attacking the object detectors, this new attack strategy influences the driving decision more effectively with less attack budgets. However, little is known on whether the revealed vulnerability remains effective in end-to-end self-driving systems and, if so, how to mitigate the threat. In this paper, we present the first systematic research on the security of object tracking in SDC. Through a comprehensive case study on the full perception pipeline of a popular open-sourced self-driving system, Baidu's Apollo, we prove the mainstream multi-object tracker (MOT) based on Kalman Filter (KF) is unsafe even with an enabled multi-sensor fusion mechanism. Our root cause analysis reveals, the vulnerability is innate to the design of KF-based MOT, which shall error-handle the prediction results from the object detectors yet the adopted KF algorithm is prone to trust the observation more when its deviation from the prediction is larger. To address this design flaw, we propose a simple yet effective security patch for KF-based MOT, the core of which is an adaptive strategy to balance the focus of KF on observations and predictions according to the anomaly index of the observation-prediction deviation, and has certified effectiveness against a generalized hijacking attack model. Extensive evaluation on 4 KF-based existing MOT implementations (including 2D and 3D, academic and Apollo ones) validate the defense effectiveness and the trivial performance overhead of our approach.