# ShapeShifter: Robust Physical Adversarial Attack on Faster R-CNN Object Detector

arXiv: https://arxiv.org/abs/1804.05810 · HTML: https://mlsanity.com/paper/1804.05810/

## Key facts

- Cited by 159 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Apr 2018.

## Built on by (most cited first, 30 of 159)

- [Automatic and Universal Prompt Injection Attacks against Large Language Models](https://mlsanity.com/paper/2403.04957/) (2024)
- [Invisible for both Camera and LiDAR: Security of Multi-Sensor Fusion based Perception in Autonomous Driving Under Physical-World Attacks](https://mlsanity.com/paper/2106.09249/) (2021)
- [FCA: Learning a 3D Full-coverage Vehicle Camouflage for Multi-view Physical Adversarial Attack](https://mlsanity.com/paper/2109.07193/) (2021)
- [Segment and Complete: Defending Object Detectors against Adversarial Patch Attacks with Robust Patch Detection](https://mlsanity.com/paper/2112.04532/) (2021)
- [Physical Adversarial Attack meets Computer Vision: A Decade Survey](https://mlsanity.com/paper/2209.15179/) (2022)
- [Black Box Adversarial Prompting for Foundation Models](https://mlsanity.com/paper/2302.04237/) (2023)
- [T-SEA: Transfer-based Self-Ensemble Attack on Object Detection](https://mlsanity.com/paper/2211.09773/) (2022)
- [X-Adv: Physical Adversarial Object Attacks against X-ray Prohibited Item Detection](https://mlsanity.com/paper/2302.09491/) (2023)
- [Physical Attack on Monocular Depth Estimation with Optimal Adversarial Patches](https://mlsanity.com/paper/2207.04718/) (2022)
- [Exploring Adversarial Robustness of Multi-Sensor Perception Systems in Self Driving](https://mlsanity.com/paper/2101.06784/) (2021)
- [Certified Patch Robustness via Smoothed Vision Transformers](https://mlsanity.com/paper/2110.07719/) (2021)
- [Robust and Accurate Object Detection via Adversarial Learning](https://mlsanity.com/paper/2103.13886/) (2021)
- [Physically Realizable Natural-Looking Clothing Textures Evade Person Detectors via 3D Modeling](https://mlsanity.com/paper/2307.01778/) (2023)
- [Improved Text Classification via Contrastive Adversarial Training](https://mlsanity.com/paper/2107.10137/) (2021)
- [Dual-Key Multimodal Backdoors for Visual Question Answering](https://mlsanity.com/paper/2112.07668/) (2021)
- [Does Physical Adversarial Example Really Matter to Autonomous Driving? Towards System-Level Effect of Adversarial Object Evasion Attack](https://mlsanity.com/paper/2308.11894/) (2023)
- [Physical Adversarial Attacks on an Aerial Imagery Object Detector](https://mlsanity.com/paper/2108.11765/) (2021)
- [Prior-Guided Adversarial Initialization for Fast Adversarial Training](https://mlsanity.com/paper/2207.08859/) (2022)
- [PAD: Patch-Agnostic Defense against Adversarial Patch Attacks](https://mlsanity.com/paper/2404.16452/) (2024)
- [Deep Learning-Based Autonomous Driving Systems: A Survey of Attacks and Defenses](https://mlsanity.com/paper/2104.01789/) (2021)
- [DetectorGuard: Provably Securing Object Detectors against Localized Patch Hiding Attacks](https://mlsanity.com/paper/2102.02956/) (2021)
- [PatchGuard++: Efficient Provable Attack Detection against Adversarial Patches](https://mlsanity.com/paper/2104.12609/) (2021)
- [Phantom Sponges: Exploiting Non-Maximum Suppression to Attack Deep Object Detectors](https://mlsanity.com/paper/2205.13618/) (2022)
- [Fooling the Eyes of Autonomous Vehicles: Robust Physical Adversarial Examples Against Traffic Sign Recognition Systems](https://mlsanity.com/paper/2201.06192/) (2022)
- [Robust Physical-World Attacks on Face Recognition](https://mlsanity.com/paper/2109.09320/) (2021)
- [A Survey on Physical Adversarial Attack in Computer Vision](https://mlsanity.com/paper/2209.14262/) (2022)
- [Evaluating Adversarial Attacks on Driving Safety in Vision-Based Autonomous Vehicles](https://mlsanity.com/paper/2108.02940/) (2021)
- [Adversarial Attacks of Vision Tasks in the Past 10 Years: A Survey](https://mlsanity.com/paper/2410.23687/) (2024)
- [QAIR: Practical Query-efficient Black-Box Attacks for Image Retrieval](https://mlsanity.com/paper/2103.02927/) (2021)
- [ObjectSeeker: Certifiably Robust Object Detection against Patch Hiding Attacks via Patch-agnostic Masking](https://mlsanity.com/paper/2202.01811/) (2022)

## Abstract (arXiv)

Given the ability to directly manipulate image pixels in the digital input space, an adversary can easily generate imperceptible perturbations to fool a Deep Neural Network (DNN) image classifier, as demonstrated in prior work. In this work, we propose ShapeShifter, an attack that tackles the more challenging problem of crafting physical adversarial perturbations to fool image-based object detectors like Faster R-CNN. Attacking an object detector is more difficult than attacking an image classifier, as it needs to mislead the classification results in multiple bounding boxes with different scales. Extending the digital attack to the physical world adds another layer of difficulty, because it requires the perturbation to be robust enough to survive real-world distortions due to different viewing distances and angles, lighting conditions, and camera limitations. We show that the Expectation over Transformation technique, which was originally proposed to enhance the robustness of adversarial perturbations in image classification, can be successfully adapted to the object detection setting. ShapeShifter can generate adversarially perturbed stop signs that are consistently mis-detected by Faster R-CNN as other objects, posing a potential threat to autonomous vehicles and other safety-critical computer vision systems.