# Attacking Motion Planners Using Adversarial Perception Errors

arXiv: https://arxiv.org/abs/2311.12722 · HTML: https://mlsanity.com/paper/2311.12722/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 47 of its 78 references that are in the map.
- First posted on arXiv 21 Nov 2023.

## Builds on (most cited first, 30 of 47)

- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [Adversarial examples in the physical world](https://mlsanity.com/paper/1607.02533/) (2016)
- [Simple Online and Realtime Tracking](https://mlsanity.com/paper/1602.00763/) (2016)
- [BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird's-Eye View Representation](https://mlsanity.com/paper/2205.13542/) (2022)
- [Adversarial Patch](https://mlsanity.com/paper/1712.09665/) (2017)
- [Decision-Based Adversarial Attacks: Reliable Attacks Against Black-Box Machine Learning Models](https://mlsanity.com/paper/1712.04248/) (2017)
- [Pyro: Deep Universal Probabilistic Programming](https://mlsanity.com/paper/1810.09538/) (2018)
- [HopSkipJumpAttack: A Query-Efficient Decision-Based Attack](https://mlsanity.com/paper/1904.02144/) (2019)
- [Emergent Complexity and Zero-shot Transfer via Unsupervised Environment Design](https://mlsanity.com/paper/2012.02096/) (2020)
- [Fooling automated surveillance cameras: adversarial patches to attack person detection](https://mlsanity.com/paper/1904.08653/) (2019)
- [Parting with Misconceptions about Learning-based Vehicle Motion Planning](https://mlsanity.com/paper/2306.07962/) (2023)
- [Efficient Decision-based Black-box Adversarial Attacks on Face Recognition](https://mlsanity.com/paper/1904.04433/) (2019)
- [Physical Adversarial Examples for Object Detectors](https://mlsanity.com/paper/1807.07769/) (2018)
- [ShapeShifter: Robust Physical Adversarial Attack on Faster R-CNN Object Detector](https://mlsanity.com/paper/1804.05810/) (2018)
- [Making an Invisibility Cloak: Real World Adversarial Attacks on Object Detectors](https://mlsanity.com/paper/1910.14667/) (2019)
- [AdvSim: Generating Safety-Critical Scenarios for Self-Driving Vehicles](https://mlsanity.com/paper/2101.06549/) (2021)
- [Query-Efficient Hard-label Black-box Attack:An Optimization-based Approach](https://mlsanity.com/paper/1807.04457/) (2018)
- [A Survey of End-to-End Driving: Architectures and Training Methods](https://mlsanity.com/paper/2003.06404/) (2020)
- [KING: Generating Safety-Critical Driving Scenarios for Robust Imitation via Kinematics Gradients](https://mlsanity.com/paper/2204.13683/) (2022)
- [On Physical Adversarial Patches for Object Detection](https://mlsanity.com/paper/1906.11897/) (2019)
- [Adaptive Stress Testing for Autonomous Vehicles](https://mlsanity.com/paper/1902.01909/) (2019)
- [A Survey of Algorithms for Black-Box Safety Validation of Cyber-Physical Systems](https://mlsanity.com/paper/2005.02979/) (2020)
- [Adversarial camera stickers: A physical camera-based attack on deep learning systems](https://mlsanity.com/paper/1904.00759/) (2019)
- [DARTS: Deceiving Autonomous Cars with Toxic Signs](https://mlsanity.com/paper/1802.06430/) (2018)
- [Learning to Evaluate Perception Models Using Planner-Centric Metrics](https://mlsanity.com/paper/2004.08745/) (2020)
- [Learning to Drive from Simulation without Real World Labels](https://mlsanity.com/paper/1812.03823/) (2018)
- [Vehicle trajectory prediction works, but not everywhere](https://mlsanity.com/paper/2112.03909/) (2021)
- [Neural Bridge Sampling for Evaluating Safety-Critical Autonomous Systems](https://mlsanity.com/paper/2008.10581/) (2020)

## Built on by (most cited first, 1 of 1)

- [EMPERROR: A Flexible Generative Perception Error Model for Probing Self-Driving Planners](https://mlsanity.com/paper/2411.07719/) (2024)

## Abstract (arXiv)

Autonomous driving (AD) systems are often built and tested in a modular fashion, where the performance of different modules is measured using task-specific metrics. These metrics should be chosen so as to capture the downstream impact of each module and the performance of the system as a whole. For example, high perception quality should enable prediction and planning to be performed safely. Even though this is true in general, we show here that it is possible to construct planner inputs that score very highly on various perception quality metrics but still lead to planning failures. In an analogy to adversarial attacks on image classifiers, we call such inputs adversarial perception errors and show they can be systematically constructed using a simple boundary-attack algorithm. We demonstrate the effectiveness of this algorithm by finding attacks for two different black-box planners in several urban and highway driving scenarios using the CARLA simulator. Finally, we analyse the properties of these attacks and show that they are isolated in the input space of the planner, and discuss their implications for AD system deployment and testing.