# Class-Aware Robust Adversarial Training for Object Detection

arXiv: https://arxiv.org/abs/2103.16148 · HTML: https://mlsanity.com/paper/2103.16148/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 33 references that are in the map.
- First posted on arXiv 30 Mar 2021.

## Builds on (most cited first, 28 of 28)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015)
- [Towards Evaluating the Robustness of Neural Networks](https://mlsanity.com/paper/1608.04644/) (2016)
- [Fast R-CNN](https://mlsanity.com/paper/1504.08083/) (2015)
- [YOLOv3: An Incremental Improvement](https://mlsanity.com/paper/1804.02767/) (2018)
- [Rich feature hierarchies for accurate object detection and semantic segmentation](https://mlsanity.com/paper/1311.2524/) (2013)
- [Adversarial examples in the physical world](https://mlsanity.com/paper/1607.02533/) (2016)
- [DeepFool: a simple and accurate method to fool deep neural networks](https://mlsanity.com/paper/1511.04599/) (2015)
- [Ensemble Adversarial Training: Attacks and Defenses](https://mlsanity.com/paper/1705.07204/) (2017)
- [Universal adversarial perturbations](https://mlsanity.com/paper/1610.08401/) (2016)
- [Distillation as a Defense to Adversarial Perturbations against Deep Neural Networks](https://mlsanity.com/paper/1511.04508/) (2015)
- [Adversarial Training for Free!](https://mlsanity.com/paper/1904.12843/) (2019)
- [Training Region-based Object Detectors with Online Hard Example Mining](https://mlsanity.com/paper/1604.03540/) (2016)
- [Defense against Adversarial Attacks Using High-Level Representation Guided Denoiser](https://mlsanity.com/paper/1712.02976/) (2017)
- [Adversarial Examples for Semantic Segmentation and Object Detection](https://mlsanity.com/paper/1703.08603/) (2017)
- [Transferable Adversarial Attacks for Image and Video Object Detection](https://mlsanity.com/paper/1811.12641/) (2018)
- [Towards Adversarially Robust Object Detection](https://mlsanity.com/paper/1907.10310/) (2019)
- [DARTS: Deceiving Autonomous Cars with Toxic Signs](https://mlsanity.com/paper/1802.06430/) (2018)
- [Maximal Jacobian-based Saliency Map Attack](https://mlsanity.com/paper/1808.07945/) (2018)
- [Robust Adversarial Perturbation on Deep Proposal-based Models](https://mlsanity.com/paper/1809.05962/) (2018)
- [ScratchDet: Training Single-Shot Object Detectors from Scratch](https://mlsanity.com/paper/1810.08425/) (2018)
- [Deep Residual Learning for Weakly-Supervised Relation Extraction](https://mlsanity.com/paper/1707.08866/) (2017)

## Built on by (most cited first, 27 of 27)

- [Advances in adversarial attacks and defenses in computer vision: A survey](https://mlsanity.com/paper/2108.00401/) (2021)
- [Segment and Complete: Defending Object Detectors against Adversarial Patch Attacks with Robust Patch Detection](https://mlsanity.com/paper/2112.04532/) (2021)
- [X-Adv: Physical Adversarial Object Attacks against X-ray Prohibited Item Detection](https://mlsanity.com/paper/2302.09491/) (2023)
- [COCO-O: A Benchmark for Object Detectors under Natural Distribution Shifts](https://mlsanity.com/paper/2307.12730/) (2023)
- [PAIF: Perception-Aware Infrared-Visible Image Fusion for Attack-Tolerant Semantic Segmentation](https://mlsanity.com/paper/2308.03979/) (2023)
- [SoK: On the Semantic AI Security in Autonomous Driving](https://mlsanity.com/paper/2203.05314/) (2022)
- [Adversarially-Aware Robust Object Detector](https://mlsanity.com/paper/2207.06202/) (2022)
- [Adversarial Training of Self-supervised Monocular Depth Estimation against Physical-World Attacks](https://mlsanity.com/paper/2301.13487/) (2023)
- [NumbOD: A Spatial-Frequency Fusion Attack Against Object Detectors](https://mlsanity.com/paper/2412.16955/) (2024)
- [On the Importance of Backbone to the Adversarial Robustness of Object Detectors](https://mlsanity.com/paper/2305.17438/) (2023)
- [A Survey and Evaluation of Adversarial Attacks for Object Detection](https://mlsanity.com/paper/2408.01934/) (2024)
- [Self-supervised Adversarial Training of Monocular Depth Estimation against Physical-World Attacks](https://mlsanity.com/paper/2406.05857/) (2024)
- [Robust and Accurate Object Detection via Self-Knowledge Distillation](https://mlsanity.com/paper/2111.07239/) (2021)
- [PBCAT: Patch-based composite adversarial training against physically realizable attacks on object detection](https://mlsanity.com/paper/2506.23581/) (2025)
- [FROD: Robust Object Detection for Free](https://mlsanity.com/paper/2308.01888/) (2023)
- [ARIA: Adversarially Robust Image Attribution for Content Provenance](https://mlsanity.com/paper/2202.12860/) (2022)
- [ScAR: Scaling Adversarial Robustness for LiDAR Object Detection](https://mlsanity.com/paper/2312.03085/) (2023)
- [A Single Set of Adversarial Clothes Breaks Multiple Defense Methods in the Physical World](https://mlsanity.com/paper/2510.17322/) (2025)
- [Diffusion Models are Robust Pretrainers](https://mlsanity.com/paper/2511.02793/) (2025)
- [Knowledge-Guided Adversarial Training for Infrared Object Detection via Thermal Radiation Modeling](https://mlsanity.com/paper/2603.25170/) (2026)
- [Cooperation or Competition: Avoiding Player Domination for Multi-Target Robustness via Adaptive Budgets](https://mlsanity.com/paper/2306.15482/) (2023)
- [Towards Generic and Controllable Attacks Against Object Detection](https://mlsanity.com/paper/2307.12342/) (2023)
- [CARNet: Collaborative Adversarial Resilience for Robust Underwater Image Enhancement and Perception](https://mlsanity.com/paper/2309.01102/) (2023)
- [Weakly Supervised Point Clouds Transformer for 3D Object Detection](https://mlsanity.com/paper/2309.04105/) (2023)
- [Unveiling the Backdoor Mechanism Hidden Behind Catastrophic Overfitting in Fast Adversarial Training](https://mlsanity.com/paper/2604.24350/) (2026)
- [LipSSD: Lipschitz-Constrained Single-Shot Detection for Adversarially Robust Object Detection](https://mlsanity.com/paper/2607.06592/) (2026)
- [Multi-Task Consistency-based Detection of Adversarial Attacks](https://mlsanity.com/paper/2608.07750/) (2026)

## Abstract (arXiv)

Object detection is an important computer vision task with plenty of real-world applications; therefore, how to enhance its robustness against adversarial attacks has emerged as a crucial issue. However, most of the previous defense methods focused on the classification task and had few analysis in the context of the object detection task. In this work, to address the issue, we present a novel class-aware robust adversarial training paradigm for the object detection task. For a given image, the proposed approach generates an universal adversarial perturbation to simultaneously attack all the occurred objects in the image through jointly maximizing the respective loss for each object. Meanwhile, instead of normalizing the total loss with the number of objects, the proposed approach decomposes the total loss into class-wise losses and normalizes each class loss using the number of objects for the class. The adversarial training based on the class weighted loss can not only balances the influence of each class but also effectively and evenly improves the adversarial robustness of trained models for all the object classes as compared with the previous defense methods. Furthermore, with the recent development of fast adversarial training, we provide a fast version of the proposed algorithm which can be trained faster than the traditional adversarial training while keeping comparable performance. With extensive experiments on the challenging PASCAL-VOC and MS-COCO datasets, the evaluation results demonstrate that the proposed defense methods can effectively enhance the robustness of the object detection models.