# Adversarial Driving: Attacking End-to-End Autonomous Driving

arXiv: https://arxiv.org/abs/2103.09151 · HTML: https://mlsanity.com/paper/2103.09151/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Placed in Adversarial scenarios and safety testing, one of 9 directions in End-to-end autonomous driving.
- Builds on 14 of its 34 references that are in the map.
- First posted on arXiv 16 Mar 2021.

## Placed in

- End-to-end autonomous driving / [Adversarial scenarios and safety testing](https://mlsanity.com/driving/adversarial-scenarios-and-safety-testing/)

## Builds on (most cited first, 14 of 14)

- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [DeepFool: a simple and accurate method to fool deep neural networks](https://mlsanity.com/paper/1511.04599/) (2015)
- [End to End Learning for Self-Driving Cars](https://mlsanity.com/paper/1604.07316/) (2016)
- [Universal adversarial perturbations](https://mlsanity.com/paper/1610.08401/) (2016)
- [Square Attack: a query-efficient black-box adversarial attack via random search](https://mlsanity.com/paper/1912.00049/) (2019)
- [AirSim: High-Fidelity Visual and Physical Simulation for Autonomous Vehicles](https://mlsanity.com/paper/1705.05065/) (2017)
- [A Survey of Autonomous Driving: Common Practices and Emerging Technologies](https://mlsanity.com/paper/1906.05113/) (2019)
- [A Survey of End-to-End Driving: Architectures and Training Methods](https://mlsanity.com/paper/2003.06404/) (2020)
- [An Analysis of Adversarial Attacks and Defenses on Autonomous Driving Models](https://mlsanity.com/paper/2002.02175/) (2020)
- [Attacking Vision-based Perception in End-to-End Autonomous Driving Models](https://mlsanity.com/paper/1910.01907/) (2019)
- [Simple Physical Adversarial Examples against End-to-End Autonomous Driving Models](https://mlsanity.com/paper/1903.05157/) (2019)
- [Adversarial Attacks, Regression, and Numerical Stability Regularization](https://mlsanity.com/paper/1812.02885/) (2018)

## Built on by (most cited first, 19 of 19)

- [Vision Language Models in Autonomous Driving: A Survey and Outlook](https://mlsanity.com/paper/2310.14414/) (2023)
- [Recent Advancements in End-to-End Autonomous Driving using Deep Learning: A Survey](https://mlsanity.com/paper/2307.04370/) (2023)
- [Mind the Gap! A Study on the Transferability of Virtual vs Physical-world Testing of Autonomous Driving Systems](https://mlsanity.com/paper/2112.11255/) (2021)
- [Sparse Adversarial Video Attacks with Spatial Transformations](https://mlsanity.com/paper/2111.05468/) (2021)
- [Reward Certification for Policy Smoothed Reinforcement Learning](https://mlsanity.com/paper/2312.06436/) (2023)
- [Boosting Adversarial Training via Fisher-Rao Norm-based Regularization](https://mlsanity.com/paper/2403.17520/) (2024)
- [Adversarial Robustness of Deep Learning: Theory, Algorithms, and Applications](https://mlsanity.com/paper/2108.10451/) (2021)
- [Safe Imitation Learning on Real-Life Highway Data for Human-like Autonomous Driving](https://mlsanity.com/paper/2110.04052/) (2021)
- [Responsible AI for Earth Observation](https://mlsanity.com/paper/2405.20868/) (2024)
- [Module-wise Adaptive Adversarial Training for End-to-end Autonomous Driving](https://mlsanity.com/paper/2409.07321/) (2024)
- [High-Frequency Anti-DreamBooth: Robust Defense against Personalized Image Synthesis](https://mlsanity.com/paper/2409.08167/) (2024)
- [End-to-End Visual Autonomous Parking via Control-Aided Attention](https://mlsanity.com/paper/2509.11090/) (2025)
- [A Human-in-the-Middle Attack against Object Detection Systems](https://mlsanity.com/paper/2208.07174/) (2022)
- [AI-Powered CPS-Enabled Vulnerable-User-Aware Urban Transportation Digital Twin: Methods and Applications](https://mlsanity.com/paper/2501.10396/) (2024)
- [Advancing Autonomous Driving System Testing: Demands, Challenges, and Future Directions](https://mlsanity.com/paper/2512.11887/) (2025)
- [Adversarial Training in Low-Label Regimes with Margin-Based Interpolation](https://mlsanity.com/paper/2411.17959/) (2024)
- [AD2: Analysis and Detection of Adversarial Threats in Visual Perception for End-to-End Autonomous Driving Systems](https://mlsanity.com/paper/2602.10160/) (2026)
- [UniAda: Universal Adaptive Multi-objective Adversarial Attack for End-to-End Autonomous Driving Systems](https://mlsanity.com/paper/2604.23362/) (2026)
- [Off the Rails: Hijacking the Scoring Head in Generative End-to-End Driving Planners with Safety-Violating Adversarial Perturbations](https://mlsanity.com/paper/2606.30807/) (2026)

## Abstract (arXiv)

As research in deep neural networks advances, deep convolutional networks become promising for autonomous driving tasks. In particular, there is an emerging trend of employing end-to-end neural network models for autonomous driving. However, previous research has shown that deep neural network classifiers are vulnerable to adversarial attacks. While for regression tasks, the effect of adversarial attacks is not as well understood. In this research, we devise two white-box targeted attacks against end-to-end autonomous driving models. Our attacks manipulate the behavior of the autonomous driving system by perturbing the input image. In an average of 800 attacks with the same attack strength (epsilon=1), the image-specific and image-agnostic attack deviates the steering angle from the original output by 0.478 and 0.111, respectively, which is much stronger than random noises that only perturbs the steering angle by 0.002 (The steering angle ranges from [-1, 1]). Both attacks can be initiated in real-time on CPUs without employing GPUs. Demo video: https://youtu.be/I0i8uN2oOP0.