# Relevance Attack on Detectors

arXiv: https://arxiv.org/abs/2008.06822 · HTML: https://mlsanity.com/paper/2008.06822/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Aug 2020.

## Built on by (most cited first, 8 of 8)

- [Adversarial Attack on Attackers: Post-Process to Mitigate Black-Box Score-Based Query Attacks](https://mlsanity.com/paper/2205.12134/) (2022)
- [A Survey and Evaluation of Adversarial Attacks for Object Detection](https://mlsanity.com/paper/2408.01934/) (2024)
- [Transferable Physical Attack against Object Detection with Separable Attention](https://mlsanity.com/paper/2205.09592/) (2022)
- [Investigating Catastrophic Overfitting in Fast Adversarial Training: A Self-fitting Perspective](https://mlsanity.com/paper/2302.11963/) (2023)
- [Query Attack by Multi-Identity Surrogates](https://mlsanity.com/paper/2105.15010/) (2021)
- [Amnesia as a Catalyst for Enhancing Black Box Pixel Attacks in Image Classification and Object Detection](https://mlsanity.com/paper/2502.07821/) (2025)
- [Unifying Gradients to Improve Real-world Robustness for Deep Networks](https://mlsanity.com/paper/2208.06228/) (2022)
- [Corrupting Attention: Evasion-Based Adversarial Attacks on Encoder Attention in Detection Transformers](https://mlsanity.com/paper/2608.06674/) (2026)

## Abstract (arXiv)

This paper focuses on high-transferable adversarial attacks on detectors, which are hard to attack in a black-box manner, because of their multiple-output characteristics and the diversity across architectures. To pursue a high attack transferability, one plausible way is to find a common property across detectors, which facilitates the discovery of common weaknesses. We are the first to suggest that the relevance map from interpreters for detectors is such a property. Based on it, we design a Relevance Attack on Detectors (RAD), which achieves a state-of-the-art transferability, exceeding existing results by above 20%. On MS COCO, the detection mAPs for all 8 black-box architectures are more than halved and the segmentation mAPs are also significantly influenced. Given the great transferability of RAD, we generate the first adversarial dataset for object detection and instance segmentation, i.e., Adversarial Objects in COntext (AOCO), which helps to quickly evaluate and improve the robustness of detectors.