# Adversarial YOLO: Defense Human Detection Patch Attacks via Detecting Adversarial Patches

arXiv: https://arxiv.org/abs/2103.08860 · HTML: https://mlsanity.com/paper/2103.08860/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 16 references that are in the map.
- First posted on arXiv 16 Mar 2021.

## Builds on (most cited first, 12 of 12)

- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [Adversarial examples in the physical world](https://mlsanity.com/paper/1607.02533/) (2016)
- [Certified Defenses against Adversarial Examples](https://mlsanity.com/paper/1801.09344/) (2018)
- [Fooling automated surveillance cameras: adversarial patches to attack person detection](https://mlsanity.com/paper/1904.08653/) (2019)
- [On Physical Adversarial Patches for Object Detection](https://mlsanity.com/paper/1906.11897/) (2019)
- [Local Gradients Smoothing: Defense against localized adversarial attacks](https://mlsanity.com/paper/1807.01216/) (2018)
- [Defending Against Physically Realizable Attacks on Image Classification](https://mlsanity.com/paper/1909.09552/) (2019)
- [Minority Reports Defense: Defending Against Adversarial Patches](https://mlsanity.com/paper/2004.13799/) (2020)
- [Role of Spatial Context in Adversarial Robustness for Object Detection](https://mlsanity.com/paper/1910.00068/) (2019)
- [PatchGuard: A Provably Robust Defense against Adversarial Patches via Small Receptive Fields and Masking](https://mlsanity.com/paper/2005.10884/) (2020)

## Built on by (most cited first, 17 of 17)

- [Segment and Complete: Defending Object Detectors against Adversarial Patch Attacks with Robust Patch Detection](https://mlsanity.com/paper/2112.04532/) (2021)
- [PatchCleanser: Certifiably Robust Defense against Adversarial Patches for Any Image Classifier](https://mlsanity.com/paper/2108.09135/) (2021)
- [PAD: Patch-Agnostic Defense against Adversarial Patch Attacks](https://mlsanity.com/paper/2404.16452/) (2024)
- [ObjectSeeker: Certifiably Robust Object Detection against Patch Hiding Attacks via Patch-agnostic Masking](https://mlsanity.com/paper/2202.01811/) (2022)
- [Physical Adversarial Attacks for Surveillance: A Survey](https://mlsanity.com/paper/2305.01074/) (2023)
- [Visual Adversarial Attacks and Defenses in the Physical World: A Survey](https://mlsanity.com/paper/2211.01671/) (2022)
- [Adversarial Examples in the Physical World: A Survey](https://mlsanity.com/paper/2311.01473/) (2023)
- [Defending Person Detection Against Adversarial Patch Attack by using Universal Defensive Frame](https://mlsanity.com/paper/2204.13004/) (2022)
- [A Survey and Evaluation of Adversarial Attacks for Object Detection](https://mlsanity.com/paper/2408.01934/) (2024)
- [We Can Always Catch You: Detecting Adversarial Patched Objects WITH or WITHOUT Signature](https://mlsanity.com/paper/2106.05261/) (2021)
- [I Don't Know You, But I Can Catch You: Real-Time Defense against Diverse Adversarial Patches for Object Detectors](https://mlsanity.com/paper/2406.10285/) (2024)
- [Defending Against Physical Adversarial Patch Attacks on Infrared Human Detection](https://mlsanity.com/paper/2309.15519/) (2023)
- [PBCAT: Patch-based composite adversarial training against physically realizable attacks on object detection](https://mlsanity.com/paper/2506.23581/) (2025)
- [Revisiting Adversarial Patch Defenses on Object Detectors: Unified Evaluation, Large-Scale Dataset, and New Insights](https://mlsanity.com/paper/2508.00649/) (2025)
- [X-Detect: Explainable Adversarial Patch Detection for Object Detectors in Retail](https://mlsanity.com/paper/2306.08422/) (2023)
- [A Single Set of Adversarial Clothes Breaks Multiple Defense Methods in the Physical World](https://mlsanity.com/paper/2510.17322/) (2025)
- [Defending against Patch-Based and Texture-Based Adversarial Attacks with Spectral Decomposition](https://mlsanity.com/paper/2604.10715/) (2026)

## Abstract (arXiv)

The security of object detection systems has attracted increasing attention, especially when facing adversarial patch attacks. Since patch attacks change the pixels in a restricted area on objects, they are easy to implement in the physical world, especially for attacking human detection systems. The existing defenses against patch attacks are mostly applied for image classification problems and have difficulty resisting human detection attacks. Towards this critical issue, we propose an efficient and effective plug-in defense component on the YOLO detection system, which we name Ad-YOLO. The main idea is to add a patch class on the YOLO architecture, which has a negligible inference increment. Thus, Ad-YOLO is expected to directly detect both the objects of interest and adversarial patches. To the best of our knowledge, our approach is the first defense strategy against human detection attacks. We investigate Ad-YOLO's performance on the YOLOv2 baseline. To improve the ability of Ad-YOLO to detect variety patches, we first use an adversarial training process to develop a patch dataset based on the Inria dataset, which we name Inria-Patch. Then, we train Ad-YOLO by a combination of Pascal VOC, Inria, and Inria-Patch datasets. With a slight drop of 0.70% mAP on VOC 2007 test set, Ad-YOLO achieves 80.31% AP of persons, which highly outperforms 33.93% AP for YOLOv2 when facing white-box patch attacks. Furthermore, compared with YOLOv2, the results facing a physical-world attack are also included to demonstrate Ad-YOLO's excellent generalization ability.