# RPATTACK: Refined Patch Attack on General Object Detectors

arXiv: https://arxiv.org/abs/2103.12469 · HTML: https://mlsanity.com/paper/2103.12469/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 14 references that are in the map.
- First posted on arXiv 23 Mar 2021.

## Builds on (most cited first, 12 of 12)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [Adversarial examples in the physical world](https://mlsanity.com/paper/1607.02533/) (2016)
- [YOLOv4: Optimal Speed and Accuracy of Object Detection](https://mlsanity.com/paper/2004.10934/) (2020)
- [Adversarial Patch](https://mlsanity.com/paper/1712.09665/) (2017)
- [Adversarial Examples for Semantic Segmentation and Object Detection](https://mlsanity.com/paper/1703.08603/) (2017)
- [DPatch: An Adversarial Patch Attack on Object Detectors](https://mlsanity.com/paper/1806.02299/) (2018)
- [Object Hider: Adversarial Patch Attack Against Object Detectors](https://mlsanity.com/paper/2010.14974/) (2020)
- [DPAttack: Diffused Patch Attacks against Universal Object Detection](https://mlsanity.com/paper/2010.11679/) (2020)

## Built on by (most cited first, 18 of 18)

- [T-SEA: Transfer-based Self-Ensemble Attack on Object Detection](https://mlsanity.com/paper/2211.09773/) (2022)
- [Towards Practical Certifiable Patch Defense with Vision Transformer](https://mlsanity.com/paper/2203.08519/) (2022)
- [Understanding the Robustness of 3D Object Detection with Bird's-Eye-View Representations in Autonomous Driving](https://mlsanity.com/paper/2303.17297/) (2023)
- [Boosting the Transferability of Adversarial Attacks with Global Momentum Initialization](https://mlsanity.com/paper/2211.11236/) (2022)
- [A Survey on Physical Adversarial Attack in Computer Vision](https://mlsanity.com/paper/2209.14262/) (2022)
- [DUAW: Data-free Universal Adversarial Watermark against Stable Diffusion Customization](https://mlsanity.com/paper/2308.09889/) (2023)
- [Query-Efficient Decision-based Black-Box Patch Attack](https://mlsanity.com/paper/2307.00477/) (2023)
- [Adversarial Patch Attacks and Defences in Vision-Based Tasks: A Survey](https://mlsanity.com/paper/2206.08304/) (2022)
- [Efficient universal shuffle attack for visual object tracking](https://mlsanity.com/paper/2203.06898/) (2022)
- [Threatening Patch Attacks on Object Detection in Optical Remote Sensing Images](https://mlsanity.com/paper/2302.06060/) (2023)
- [Explicit and Implicit Knowledge Distillation via Unlabeled Data](https://mlsanity.com/paper/2302.08771/) (2023)
- [A Survey and Evaluation of Adversarial Attacks for Object Detection](https://mlsanity.com/paper/2408.01934/) (2024)
- [A Comprehensive Study on the Robustness of Image Classification and Object Detection in Remote Sensing: Surveying and Benchmarking](https://mlsanity.com/paper/2306.12111/) (2023)
- [DOEPatch: Dynamically Optimized Ensemble Model for Adversarial Patches Generation](https://mlsanity.com/paper/2312.16907/) (2023)
- [AdvLogo: Adversarial Patch Attack against Object Detectors based on Diffusion Models](https://mlsanity.com/paper/2409.07002/) (2024)
- [Adversarial Examples in Environment Perception for Automated Driving (Review)](https://mlsanity.com/paper/2504.08414/) (2025)
- [Structured Universal Adversarial Attacks on Object Detection for Video Sequences](https://mlsanity.com/paper/2510.14460/) (2025)
- [The Unseen Adversaries: Robust and Generalized Defense Against Adversarial Patches](https://mlsanity.com/paper/2604.26317/) (2026)

## Abstract (arXiv)

Nowadays, general object detectors like YOLO and Faster R-CNN as well as their variants are widely exploited in many applications. Many works have revealed that these detectors are extremely vulnerable to adversarial patch attacks. The perturbed regions generated by previous patch-based attack works on object detectors are very large which are not necessary for attacking and perceptible for human eyes. To generate much less but more efficient perturbation, we propose a novel patch-based method for attacking general object detectors. Firstly, we propose a patch selection and refining scheme to find the pixels which have the greatest importance for attack and remove the inconsequential perturbations gradually. Then, for a stable ensemble attack, we balance the gradients of detectors to avoid over-optimizing one of them during the training phase. Our RPAttack can achieve an amazing missed detection rate of 100% for both Yolo v4 and Faster R-CNN while only modifies 0.32% pixels on VOC 2007 test set. Our code is available at https://github.com/VDIGPKU/RPAttack.