# Fooling the Eyes of Autonomous Vehicles: Robust Physical Adversarial Examples Against Traffic Sign Recognition Systems

arXiv: https://arxiv.org/abs/2201.06192 · HTML: https://mlsanity.com/paper/2201.06192/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Builds on 46 of its 66 references that are in the map.
- First posted on arXiv 17 Jan 2022.

## Builds on (most cited first, 30 of 46)

- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015)
- [Towards Evaluating the Robustness of Neural Networks](https://mlsanity.com/paper/1608.04644/) (2016)
- [YOLOv3: An Incremental Improvement](https://mlsanity.com/paper/1804.02767/) (2018)
- [Adversarial examples in the physical world](https://mlsanity.com/paper/1607.02533/) (2016)
- [YOLOv4: Optimal Speed and Accuracy of Object Detection](https://mlsanity.com/paper/2004.10934/) (2020)
- [Objects as Points](https://mlsanity.com/paper/1904.07850/) (2019)
- [Ensemble Adversarial Training: Attacks and Defenses](https://mlsanity.com/paper/1705.07204/) (2017)
- [Path Aggregation Network for Instance Segmentation](https://mlsanity.com/paper/1803.01534/) (2018)
- [The Limitations of Deep Learning in Adversarial Settings](https://mlsanity.com/paper/1511.07528/) (2015)
- [Fast is better than free: Revisiting adversarial training](https://mlsanity.com/paper/2001.03994/) (2020)
- [Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition](https://mlsanity.com/paper/1406.4729/) (2014)
- [Distillation as a Defense to Adversarial Perturbations against Deep Neural Networks](https://mlsanity.com/paper/1511.04508/) (2015)
- [Countering Adversarial Images using Input Transformations](https://mlsanity.com/paper/1711.00117/) (2017)
- [Feature Squeezing: Detecting Adversarial Examples in Deep Neural Networks](https://mlsanity.com/paper/1704.01155/) (2017)
- [CSPNet: A New Backbone that can Enhance Learning Capability of CNN](https://mlsanity.com/paper/1911.11929/) (2019)
- [Defense-GAN: Protecting Classifiers Against Adversarial Attacks Using Generative Models](https://mlsanity.com/paper/1805.06605/) (2018)
- [Defense against Adversarial Attacks Using High-Level Representation Guided Denoiser](https://mlsanity.com/paper/1712.02976/) (2017)
- [Adversarial Attacks on Neural Network Policies](https://mlsanity.com/paper/1702.02284/) (2017)
- [Feature Denoising for Improving Adversarial Robustness](https://mlsanity.com/paper/1812.03411/) (2018)
- [PixelDefend: Leveraging Generative Models to Understand and Defend against Adversarial Examples](https://mlsanity.com/paper/1710.10766/) (2017)
- [On Detecting Adversarial Perturbations](https://mlsanity.com/paper/1702.04267/) (2017)
- [Physical Adversarial Examples for Object Detectors](https://mlsanity.com/paper/1807.07769/) (2018)
- [ShapeShifter: Robust Physical Adversarial Attack on Faster R-CNN Object Detector](https://mlsanity.com/paper/1804.05810/) (2018)
- [Adversarially Robust Distillation](https://mlsanity.com/paper/1905.09747/) (2019)
- [Communication-Efficient Federated Deep Learning with Asynchronous Model Update and Temporally Weighted Aggregation](https://mlsanity.com/paper/1903.07424/) (2019)

## Built on by (most cited first, 27 of 27)

- [Does Physical Adversarial Example Really Matter to Autonomous Driving? Towards System-Level Effect of Adversarial Object Evasion Attack](https://mlsanity.com/paper/2308.11894/) (2023)
- [A Survey on Physical Adversarial Attack in Computer Vision](https://mlsanity.com/paper/2209.14262/) (2022)
- [Invisible Reflections: Leveraging Infrared Laser Reflections to Target Traffic Sign Perception](https://mlsanity.com/paper/2401.03582/) (2024)
- [Adversarial Attacks on Traffic Sign Recognition: A Survey](https://mlsanity.com/paper/2307.08278/) (2023)
- [Breaking Barriers in Physical-World Adversarial Examples: Improving Robustness and Transferability via Robust Feature](https://mlsanity.com/paper/2412.16958/) (2024)
- [Dynamic Adversarial Attacks on Autonomous Driving Systems](https://mlsanity.com/paper/2312.06701/) (2023)
- [Self-Deception: Reverse Penetrating the Semantic Firewall of Large Language Models](https://mlsanity.com/paper/2308.11521/) (2023)
- [ControlLoc: Physical-World Hijacking Attack on Visual Perception in Autonomous Driving](https://mlsanity.com/paper/2406.05810/) (2024)
- [Revisiting Physical-World Adversarial Attack on Traffic Sign Recognition: A Commercial Systems Perspective](https://mlsanity.com/paper/2409.09860/) (2024)
- [A Comprehensive Study on the Robustness of Image Classification and Object Detection in Remote Sensing: Surveying and Benchmarking](https://mlsanity.com/paper/2306.12111/) (2023)
- [Secure Traffic Sign Recognition: An Attention-Enabled Universal Image Inpainting Mechanism against Light Patch Attacks](https://mlsanity.com/paper/2409.04133/) (2024)
- [SlowPerception: Physical-World Latency Attack against Visual Perception in Autonomous Driving](https://mlsanity.com/paper/2406.05800/) (2024)
- [The Fluorescent Veil: A Stealthy and Effective Physical Adversarial Patch Against Traffic Sign Recognition](https://mlsanity.com/paper/2409.12394/) (2024)
- [Flashy Backdoor: Real-world Environment Backdoor Attack on SNNs with DVS Cameras](https://mlsanity.com/paper/2411.03022/) (2024)
- [Why Don't You Clean Your Glasses? Perception Attacks with Dynamic Optical Perturbations](https://mlsanity.com/paper/2307.13131/) (2023)
- [3DGAA: Realistic and Robust 3D Gaussian-based Adversarial Attack for Autonomous Driving](https://mlsanity.com/paper/2507.09993/) (2025)
- [Mitigation of Camouflaged Adversarial Attacks in Autonomous Vehicles--A Case Study Using CARLA Simulator](https://mlsanity.com/paper/2502.05208/) (2025)
- [Toward Inherently Robust VLMs Against Visual Perception Attacks](https://mlsanity.com/paper/2506.11472/) (2025)
- [Beyond Crash: Hijacking Your Autonomous Vehicle for Fun and Profit](https://mlsanity.com/paper/2602.07249/) (2026)
- [Towards Generic and Controllable Attacks Against Object Detection](https://mlsanity.com/paper/2307.12342/) (2023)
- [Redesigning Traffic Signs to Mitigate Machine-Learning Patch Attacks](https://mlsanity.com/paper/2402.04660/) (2024)
- [Towards an End-to-End (E2E) Adversarial Learning and Application in the Physical World](https://mlsanity.com/paper/2501.08258/) (2025)
- [T2I-Based Physical-World Appearance Attack against Traffic Sign Recognition Systems in Autonomous Driving](https://mlsanity.com/paper/2511.12956/) (2025)
- [Robust Physical Adversarial Patches Using Dynamically Optimized Clusters](https://mlsanity.com/paper/2511.18656/) (2025)
- [Physical Evaluation of Naturalistic Adversarial Patches for Camera-Based Traffic-Sign Detection](https://mlsanity.com/paper/2603.00217/) (2026)
- [AdROD: HyperNetwork-based Adversarially Robust Object Detection for Autonomous Driving](https://mlsanity.com/paper/2608.16031/) (2026)
- [Adversarial Calibration Attack on Autonomous Vehicles](https://mlsanity.com/paper/2608.28778/) (2026)

## Abstract (arXiv)

Adversarial Examples (AEs) can deceive Deep Neural Networks (DNNs) and have received a lot of attention recently. However, majority of the research on AEs is in the digital domain and the adversarial patches are static, which is very different from many real-world DNN applications such as Traffic Sign Recognition (TSR) systems in autonomous vehicles. In TSR systems, object detectors use DNNs to process streaming video in real time. From the view of object detectors, the traffic sign`s position and quality of the video are continuously changing, rendering the digital AEs ineffective in the physical world. In this paper, we propose a systematic pipeline to generate robust physical AEs against real-world object detectors. Robustness is achieved in three ways. First, we simulate the in-vehicle cameras by extending the distribution of image transformations with the blur transformation and the resolution transformation. Second, we design the single and multiple bounding boxes filters to improve the efficiency of the perturbation training. Third, we consider four representative attack vectors, namely Hiding Attack, Appearance Attack, Non-Target Attack and Target Attack. We perform a comprehensive set of experiments under a variety of environmental conditions, and considering illuminations in sunny and cloudy weather as well as at night. The experimental results show that the physical AEs generated from our pipeline are effective and robust when attacking the YOLO v5 based TSR system. The attacks have good transferability and can deceive other state-of-the-art object detectors. We launched HA and NTA on a brand-new 2021 model vehicle. Both attacks are successful in fooling the TSR system, which could be a life-threatening case for autonomous vehicles. Finally, we discuss three defense mechanisms based on image preprocessing, AEs detection, and model enhancing.