# Context-Aware Transfer Attacks for Object Detection

arXiv: https://arxiv.org/abs/2112.03223 · HTML: https://mlsanity.com/paper/2112.03223/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 55 references that are in the map.
- First posted on arXiv 6 Dec 2021.

## Builds on (most cited first, 30 of 37)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [Towards Evaluating the Robustness of Neural Networks](https://mlsanity.com/paper/1608.04644/) (2016)
- [YOLOv3: An Incremental Improvement](https://mlsanity.com/paper/1804.02767/) (2018)
- [MMDetection: Open MMLab Detection Toolbox and Benchmark](https://mlsanity.com/paper/1906.07155/) (2019)
- [DeepFool: a simple and accurate method to fool deep neural networks](https://mlsanity.com/paper/1511.04599/) (2015)
- [Practical Black-Box Attacks against Machine Learning](https://mlsanity.com/paper/1602.02697/) (2016)
- [ZOO: Zeroth Order Optimization based Black-box Attacks to Deep Neural Networks without Training Substitute Models](https://mlsanity.com/paper/1708.03999/) (2017)
- [Improving Transferability of Adversarial Examples with Input Diversity](https://mlsanity.com/paper/1803.06978/) (2018)
- [Delving into Transferable Adversarial Examples and Black-box Attacks](https://mlsanity.com/paper/1611.02770/) (2016)
- [Decision-Based Adversarial Attacks: Reliable Attacks Against Black-Box Machine Learning Models](https://mlsanity.com/paper/1712.04248/) (2017)
- [Nesterov Accelerated Gradient and Scale Invariance for Adversarial Attacks](https://mlsanity.com/paper/1908.06281/) (2019)
- [HopSkipJumpAttack: A Query-Efficient Decision-Based Attack](https://mlsanity.com/paper/1904.02144/) (2019)
- [Adversarial Examples for Semantic Segmentation and Object Detection](https://mlsanity.com/paper/1703.08603/) (2017)
- [Libra R-CNN: Towards Balanced Learning for Object Detection](https://mlsanity.com/paper/1904.02701/) (2019)
- [Simple Black-box Adversarial Attacks](https://mlsanity.com/paper/1905.07121/) (2019)
- [Enhancing the Transferability of Adversarial Attacks through Variance Tuning](https://mlsanity.com/paper/2103.15571/) (2021)
- [Making an Invisibility Cloak: Real World Adversarial Attacks on Object Detectors](https://mlsanity.com/paper/1910.14667/) (2019)
- [DPatch: An Adversarial Patch Attack on Object Detectors](https://mlsanity.com/paper/1806.02299/) (2018)
- [Improving Black-box Adversarial Attacks with a Transfer-based Prior](https://mlsanity.com/paper/1906.06919/) (2019)
- [Transferable Adversarial Attacks for Image and Video Object Detection](https://mlsanity.com/paper/1811.12641/) (2018)
- [QEBA: Query-Efficient Boundary-Based Blackbox Attack](https://mlsanity.com/paper/2005.14137/) (2020)
- [FreeAnchor: Learning to Match Anchors for Visual Object Detection](https://mlsanity.com/paper/1909.02466/) (2019)
- [Inside-Outside Net: Detecting Objects in Context with Skip Pooling and Recurrent Neural Networks](https://mlsanity.com/paper/1512.04143/) (2015)
- [Black-Box Adversarial Attack with Transferable Model-based Embedding](https://mlsanity.com/paper/1911.07140/) (2019)
- [Robust Object Detection under Occlusion with Context-Aware CompositionalNets](https://mlsanity.com/paper/2005.11643/) (2020)

## Built on by (most cited first, 18 of 18)

- [Physical Adversarial Attack meets Computer Vision: A Decade Survey](https://mlsanity.com/paper/2209.15179/) (2022)
- [A Survey on Transferability of Adversarial Examples across Deep Neural Networks](https://mlsanity.com/paper/2310.17626/) (2023)
- [HOTCOLD Block: Fooling Thermal Infrared Detectors with a Novel Wearable Design](https://mlsanity.com/paper/2212.05709/) (2022)
- [Zero-Query Transfer Attacks on Context-Aware Object Detectors](https://mlsanity.com/paper/2203.15230/) (2022)
- [Ensemble-based Blackbox Attacks on Dense Prediction](https://mlsanity.com/paper/2303.14304/) (2023)
- [Blackbox Attacks via Surrogate Ensemble Search](https://mlsanity.com/paper/2208.03610/) (2022)
- [GAMA: Generative Adversarial Multi-Object Scene Attacks](https://mlsanity.com/paper/2209.09502/) (2022)
- [Overload: Latency Attacks on Object Detection for Edge Devices](https://mlsanity.com/paper/2304.05370/) (2023)
- [A Survey and Evaluation of Adversarial Attacks for Object Detection](https://mlsanity.com/paper/2408.01934/) (2024)
- [Adversarial Attacks on Event-Based Pedestrian Detectors: A Physical Approach](https://mlsanity.com/paper/2503.00377/) (2025)
- [Exploring Adversarial Transferability between Kolmogorov-arnold Networks](https://mlsanity.com/paper/2503.06276/) (2025)
- [Steal Now and Attack Later: Evaluating Robustness of Object Detection against Black-box Adversarial Attacks](https://mlsanity.com/paper/2404.15881/) (2024)
- [GLOW: Global Layout Aware Attacks on Object Detection](https://mlsanity.com/paper/2302.14166/) (2023)
- [One Object, Multiple Lies: A Benchmark for Cross-task Adversarial Attack on Unified Vision-Language Models](https://mlsanity.com/paper/2507.07709/) (2025)
- [Out-of-the-box: Black-box Causal Attacks on Object Detectors](https://mlsanity.com/paper/2512.03730/) (2025)
- [Object-fabrication Targeted Attack for Object Detection](https://mlsanity.com/paper/2212.06431/) (2022)
- [On Feasibility of Intent Obfuscating Attacks](https://mlsanity.com/paper/2408.02674/) (2024)
- [A Unified Spatial Alignment Framework for Highly Transferable Transformation-Based Attacks on Spatially Structured Tasks](https://mlsanity.com/paper/2603.25230/) (2026)

## Abstract (arXiv)

Blackbox transfer attacks for image classifiers have been extensively studied in recent years. In contrast, little progress has been made on transfer attacks for object detectors. Object detectors take a holistic view of the image and the detection of one object (or lack thereof) often depends on other objects in the scene. This makes such detectors inherently context-aware and adversarial attacks in this space are more challenging than those targeting image classifiers. In this paper, we present a new approach to generate context-aware attacks for object detectors. We show that by using co-occurrence of objects and their relative locations and sizes as context information, we can successfully generate targeted mis-categorization attacks that achieve higher transfer success rates on blackbox object detectors than the state-of-the-art. We test our approach on a variety of object detectors with images from PASCAL VOC and MS COCO datasets and demonstrate up to 20 percentage points improvement in performance compared to the other state-of-the-art methods.