Attacking Motion Planners Using Adversarial Perception Errors
Not on the map. Why?
Abstract (arXiv)
Autonomous driving (AD) systems are often built and tested in a modular fashion, where the performance of different modules is measured using task-specific metrics. These metrics should be chosen so as to capture the downstream impact of each module and the performance of the system as a whole. For example, high perception quality should enable prediction and planning to be performed safely. Even though this is true in general, we show here that it is possible to construct planner inputs that score very highly on various perception quality metrics but still lead to planning failures. In an analogy to adversarial attacks on image classifiers, we call such inputs adversarial perception errors and show they can be systematically constructed using a simple boundary-attack algorithm. We demonstrate the effectiveness of this algorithm by finding attacks for two different black-box planners in several urban and highway driving scenarios using the CARLA simulator. Finally, we analyse the properties of these attacks and show that they are isolated in the input space of the planner, and discuss their implications for AD system deployment and testing.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 47 of its 78 references that are in the map. First posted on arXiv 21 Nov 2023.
Builds on
47 of 78 references.
References · 47
- 2013Intriguing properties of neural networkscited by 5,155
- 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
- 2017CARLA: An Open Urban Driving Simulatorcited by 2,449
- 2016Adversarial examples in the physical worldcited by 1,769
- 2016Simple Online and Realtime Trackingcited by 868
- 2022BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird's-Eye View Representationcited by 850
- 2017Adversarial Patchcited by 517
- 2017Decision-Based Adversarial Attacks: Reliable Attacks Against Black-Box Machine Learning Modelscited by 462
- 2018Pyro: Deep Universal Probabilistic Programmingcited by 380
- 2019HopSkipJumpAttack: A Query-Efficient Decision-Based Attackcited by 333
- 2020Emergent Complexity and Zero-shot Transfer via Unsupervised Environment Designcited by 275
- 2019Fooling automated surveillance cameras: adversarial patches to attack person detectioncited by 268
35 more
- 2023Parting with Misconceptions about Learning-based Vehicle Motion Planningcited by 242
- 2019Efficient Decision-based Black-box Adversarial Attacks on Face Recognitioncited by 175
- 2018Physical Adversarial Examples for Object Detectorscited by 163
- 2018ShapeShifter: Robust Physical Adversarial Attack on Faster R-CNN Object Detectorcited by 159
- 2019Making an Invisibility Cloak: Real World Adversarial Attacks on Object Detectorscited by 155
- 2021AdvSim: Generating Safety-Critical Scenarios for Self-Driving Vehiclescited by 137
- 2018Query-Efficient Hard-label Black-box Attack:An Optimization-based Approachcited by 130
- 2020A Survey of End-to-End Driving: Architectures and Training Methodscited by 123
- 2022KING: Generating Safety-Critical Driving Scenarios for Robust Imitation via Kinematics Gradientscited by 96
- 2019On Physical Adversarial Patches for Object Detectioncited by 93
- 2019Adaptive Stress Testing for Autonomous Vehiclescited by 71
- 2020A Survey of Algorithms for Black-Box Safety Validation of Cyber-Physical Systemscited by 71
- 2019Adversarial camera stickers: A physical camera-based attack on deep learning systemscited by 70
- 2018DARTS: Deceiving Autonomous Cars with Toxic Signscited by 54
- 2020Learning to Evaluate Perception Models Using Planner-Centric Metricscited by 49
- 2018Learning to Drive from Simulation without Real World Labelscited by 39
- 2021Vehicle trajectory prediction works, but not everywherecited by 39
- 2020Neural Bridge Sampling for Evaluating Safety-Critical Autonomous Systemscited by 35
- 2019Adaptive Stress Testing with Reward Augmentation for Autonomous Vehicle Validationcited by 33
- 2019Fooling a Real Car with Adversarial Traffic Signscited by 29
- 2019Sim2Real Predictivity: Does Evaluation in Simulation Predict Real-World Performance?cited by 26
- 2019LiDAR Sensor modeling and Data augmentation with GANs for Autonomous drivingcited by 22
- 2022CausalAgents: A Robustness Benchmark for Motion Forecasting using Causal Relationshipscited by 18
- 2019Efficient Autonomy Validation in Simulation with Adaptive Stress Testingcited by 13
- 2021Finding Critical Scenarios for Automated Driving Systems: A Systematic Literature Reviewcited by 12
- 2021Active learning for distributionally robust level-set estimationcited by 10
- 2020A Two-Stage Optimization-based Motion Planner for Safe Urban Drivingcited by 9
- 2020Adversarial Attacks on Optimization based Plannerscited by 8
- 2022Testing Rare Downstream Safety Violations via Upstream Adaptive Sampling of Perception Error Modelscited by 7
- 2021On Determinism of Game Engines used for Simulation-based Autonomous Vehicle Verificationcited by 6
- 2022Preprocessors Matter! Realistic Decision-Based Attacks on Machine Learning Systemscited by 5
- 2021A Step Towards Efficient Evaluation of Complex Perception Tasks in Simulationcited by 4
- 2022Risk-Driven Design of Perception Systemscited by 4
- 2022Validation of Composite Systems by Discrepancy Propagationcited by 2
- 2022An Active Learning Reliability Method for Systems with Partially Defined Performance Functionscited by 1
Built on by
Citing papers per year: 2024: 1.
1 paper.