# Segment and Complete: Defending Object Detectors against Adversarial Patch Attacks with Robust Patch Detection

arXiv: https://arxiv.org/abs/2112.04532 · HTML: https://mlsanity.com/paper/2112.04532/

## Key facts

- Cited by 67 papers in the map (data through 14 Oct 2026).
- Builds on 39 of its 54 references that are in the map.
- First posted on arXiv 8 Dec 2021.

## Builds on (most cited first, 30 of 39)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015)
- [Fast R-CNN](https://mlsanity.com/paper/1504.08083/) (2015)
- [Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation](https://mlsanity.com/paper/1308.3432/) (2013)
- [YOLO9000: Better, Faster, Stronger](https://mlsanity.com/paper/1612.08242/) (2016)
- [Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples](https://mlsanity.com/paper/1802.00420/) (2018)
- [Adversarial Patch](https://mlsanity.com/paper/1712.09665/) (2017)
- [Feature Squeezing: Detecting Adversarial Examples in Deep Neural Networks](https://mlsanity.com/paper/1704.01155/) (2017)
- [Fooling automated surveillance cameras: adversarial patches to attack person detection](https://mlsanity.com/paper/1904.08653/) (2019)
- [A study of the effect of JPG compression on adversarial images](https://mlsanity.com/paper/1608.00853/) (2016)
- [Physical Adversarial Examples for Object Detectors](https://mlsanity.com/paper/1807.07769/) (2018)
- [ShapeShifter: Robust Physical Adversarial Attack on Faster R-CNN Object Detector](https://mlsanity.com/paper/1804.05810/) (2018)
- [Making an Invisibility Cloak: Real World Adversarial Attacks on Object Detectors](https://mlsanity.com/paper/1910.14667/) (2019)
- [DPatch: An Adversarial Patch Attack on Object Detectors](https://mlsanity.com/paper/1806.02299/) (2018)
- [xView: Objects in Context in Overhead Imagery](https://mlsanity.com/paper/1802.07856/) (2018)
- [On the Robustness of Semantic Segmentation Models to Adversarial Attacks](https://mlsanity.com/paper/1711.09856/) (2017)
- [LaVAN: Localized and Visible Adversarial Noise](https://mlsanity.com/paper/1801.02608/) (2018)
- [Certified Defenses for Adversarial Patches](https://mlsanity.com/paper/2003.06693/) (2020)
- [(De)Randomized Smoothing for Certifiable Defense against Patch Attacks](https://mlsanity.com/paper/2002.10733/) (2020)
- [On Physical Adversarial Patches for Object Detection](https://mlsanity.com/paper/1906.11897/) (2019)
- [Local Gradients Smoothing: Defense against localized adversarial attacks](https://mlsanity.com/paper/1807.01216/) (2018)
- [Defending Against Physically Realizable Attacks on Image Classification](https://mlsanity.com/paper/1909.09552/) (2019)
- [Towards Adversarially Robust Object Detection](https://mlsanity.com/paper/1907.10310/) (2019)
- [Adversarial Training against Location-Optimized Adversarial Patches](https://mlsanity.com/paper/2005.02313/) (2020)
- [Detection as Regression: Certified Object Detection by Median Smoothing](https://mlsanity.com/paper/2007.03730/) (2020)

## Built on by (most cited first, 30 of 67)

- [Towards Robust and Secure Embodied AI: A Survey on Vulnerabilities and Attacks](https://mlsanity.com/paper/2502.13175/) (2025)
- [PAD: Patch-Agnostic Defense against Adversarial Patch Attacks](https://mlsanity.com/paper/2404.16452/) (2024)
- [PatchZero: Defending against Adversarial Patch Attacks by Detecting and Zeroing the Patch](https://mlsanity.com/paper/2207.01795/) (2022)
- [Effectiveness Assessment of Recent Large Vision-Language Models](https://mlsanity.com/paper/2403.04306/) (2024)
- [Instruct2Attack: Language-Guided Semantic Adversarial Attacks](https://mlsanity.com/paper/2311.15551/) (2023)
- [DIFFender: Diffusion-Based Adversarial Defense against Patch Attacks](https://mlsanity.com/paper/2306.09124/) (2023)
- [A White-Box False Positive Adversarial Attack Method on Contrastive Loss Based Offline Handwritten Signature Verification Models](https://mlsanity.com/paper/2308.08925/) (2023)
- [Visual Adversarial Attacks and Defenses in the Physical World: A Survey](https://mlsanity.com/paper/2211.01671/) (2022)
- [PatchCURE: Improving Certifiable Robustness, Model Utility, and Computation Efficiency of Adversarial Patch Defenses](https://mlsanity.com/paper/2310.13076/) (2023)
- [Interpolated Joint Space Adversarial Training for Robust and Generalizable Defenses](https://mlsanity.com/paper/2112.06323/) (2021)
- [Defending From Physically-Realizable Adversarial Attacks Through Internal Over-Activation Analysis](https://mlsanity.com/paper/2203.07341/) (2022)
- [Diffusion to Confusion: Naturalistic Adversarial Patch Generation Based on Diffusion Model for Object Detector](https://mlsanity.com/paper/2307.08076/) (2023)
- [Adversarial Examples in the Physical World: A Survey](https://mlsanity.com/paper/2311.01473/) (2023)
- [Artwork Protection Against Neural Style Transfer Using Locally Adaptive Adversarial Color Attack](https://mlsanity.com/paper/2401.09673/) (2024)
- [Towards Improved Input Masking for Convolutional Neural Networks](https://mlsanity.com/paper/2211.14646/) (2022)
- [A Survey and Evaluation of Adversarial Attacks for Object Detection](https://mlsanity.com/paper/2408.01934/) (2024)
- [Detection Defenses: An Empty Promise against Adversarial Patch Attacks on Optical Flow](https://mlsanity.com/paper/2310.17403/) (2023)
- [A Satellite Imagery Dataset for Long-Term Sustainable Development in United States Cities](https://mlsanity.com/paper/2308.00465/) (2023)
- [I Don't Know You, But I Can Catch You: Real-Time Defense against Diverse Adversarial Patches for Object Detectors](https://mlsanity.com/paper/2406.10285/) (2024)
- [AdvLogo: Adversarial Patch Attack against Object Detectors based on Diffusion Models](https://mlsanity.com/paper/2409.07002/) (2024)
- [Real-world Adversarial Defense against Patch Attacks based on Diffusion Model](https://mlsanity.com/paper/2409.09406/) (2024)
- [Don't Lag, RAG: Training-Free Adversarial Detection Using RAG](https://mlsanity.com/paper/2504.04858/) (2025)
- [Embodied Active Defense: Leveraging Recurrent Feedback to Counter Adversarial Patches](https://mlsanity.com/paper/2404.00540/) (2024)
- [RS-Reg: Probabilistic and Robust Certified Regression Through Randomized Smoothing](https://mlsanity.com/paper/2405.08892/) (2024)
- [Self-Supervised Representation Learning for Adversarial Attack Detection](https://mlsanity.com/paper/2407.04382/) (2024)
- [Natural Language Induced Adversarial Images](https://mlsanity.com/paper/2410.08620/) (2024)
- [BadPatch: Diffusion-Based Generation of Physical Adversarial Patches](https://mlsanity.com/paper/2412.01440/) (2024)
- [Distributional Modeling for Location-Aware Adversarial Patches](https://mlsanity.com/paper/2306.16131/) (2023)
- [Attention-Based Real-Time Defenses for Physical Adversarial Attacks in Vision Applications](https://mlsanity.com/paper/2311.11191/) (2023)
- [RADAP: A Robust and Adaptive Defense Against Diverse Adversarial Patches on Face Recognition](https://mlsanity.com/paper/2311.17339/) (2023)

## Abstract (arXiv)

Object detection plays a key role in many security-critical systems. Adversarial patch attacks, which are easy to implement in the physical world, pose a serious threat to state-of-the-art object detectors. Developing reliable defenses for object detectors against patch attacks is critical but severely understudied. In this paper, we propose Segment and Complete defense (SAC), a general framework for defending object detectors against patch attacks through detection and removal of adversarial patches. We first train a patch segmenter that outputs patch masks which provide pixel-level localization of adversarial patches. We then propose a self adversarial training algorithm to robustify the patch segmenter. In addition, we design a robust shape completion algorithm, which is guaranteed to remove the entire patch from the images if the outputs of the patch segmenter are within a certain Hamming distance of the ground-truth patch masks. Our experiments on COCO and xView datasets demonstrate that SAC achieves superior robustness even under strong adaptive attacks with no reduction in performance on clean images, and generalizes well to unseen patch shapes, attack budgets, and unseen attack methods. Furthermore, we present the APRICOT-Mask dataset, which augments the APRICOT dataset with pixel-level annotations of adversarial patches. We show SAC can significantly reduce the targeted attack success rate of physical patch attacks. Our code is available at https://github.com/joellliu/SegmentAndComplete.