# Can't Slow me Down: Learning Robust and Hardware-Adaptive Object Detectors against Latency Attacks for Edge Devices

arXiv: https://arxiv.org/abs/2412.02171 · HTML: https://mlsanity.com/paper/2412.02171/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 52 references that are in the map.
- First posted on arXiv 3 Dec 2024.

## Builds on (most cited first, 30 of 33)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015)
- [YOLOv3: An Incremental Improvement](https://mlsanity.com/paper/1804.02767/) (2018)
- [Rich feature hierarchies for accurate object detection and semantic segmentation](https://mlsanity.com/paper/1311.2524/) (2013)
- [Theoretically Principled Trade-off between Robustness and Accuracy](https://mlsanity.com/paper/1901.08573/) (2019)
- [Generalized Intersection over Union: A Metric and A Loss for Bounding Box Regression](https://mlsanity.com/paper/1902.09630/) (2019)
- [BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learning](https://mlsanity.com/paper/1805.04687/) (2018)
- [DETRs Beat YOLOs on Real-time Object Detection](https://mlsanity.com/paper/2304.08069/) (2023)
- [Object Detection in 20 Years: A Survey](https://mlsanity.com/paper/1905.05055/) (2019)
- [DiffusionDet: Diffusion Model for Object Detection](https://mlsanity.com/paper/2211.09788/) (2022)
- [Adversarial Examples for Semantic Segmentation and Object Detection](https://mlsanity.com/paper/1703.08603/) (2017)
- [Understanding Robustness of Transformers for Image Classification](https://mlsanity.com/paper/2103.14586/) (2021)
- [Learning non-maximum suppression](https://mlsanity.com/paper/1705.02950/) (2017)
- [Sponge Examples: Energy-Latency Attacks on Neural Networks](https://mlsanity.com/paper/2006.03463/) (2020)
- [Dynamic R-CNN: Towards High Quality Object Detection via Dynamic Training](https://mlsanity.com/paper/2004.06002/) (2020)
- [Enhancing Geometric Factors in Model Learning and Inference for Object Detection and Instance Segmentation](https://mlsanity.com/paper/2005.03572/) (2020)
- [Towards Adversarially Robust Object Detection](https://mlsanity.com/paper/1907.10310/) (2019)
- [Distilling Robust and Non-Robust Features in Adversarial Examples by Information Bottleneck](https://mlsanity.com/paper/2204.02735/) (2022)
- [Wise-IoU: Bounding Box Regression Loss with Dynamic Focusing Mechanism](https://mlsanity.com/paper/2301.10051/) (2023)
- [Phantom Sponges: Exploiting Non-Maximum Suppression to Attack Deep Object Detectors](https://mlsanity.com/paper/2205.13618/) (2022)
- [Robust Adversarial Perturbation on Deep Proposal-based Models](https://mlsanity.com/paper/1809.05962/) (2018)
- [AIDE: An Automatic Data Engine for Object Detection in Autonomous Driving](https://mlsanity.com/paper/2403.17373/) (2024)
- [Adversarial Attack and Defense of YOLO Detectors in Autonomous Driving Scenarios](https://mlsanity.com/paper/2202.04781/) (2022)
- [Alpha-IoU: A Family of Power Intersection over Union Losses for Bounding Box Regression](https://mlsanity.com/paper/2110.13675/) (2021)
- [Balance, Imbalance, and Rebalance: Understanding Robust Overfitting from a Minimax Game Perspective](https://mlsanity.com/paper/2310.19360/) (2023)

## Built on by (most cited first, 3 of 3)

- [RemedyGS: Defend 3D Gaussian Splatting against Computation Cost Attacks](https://mlsanity.com/paper/2511.22147/) (2025)
- [Rethinking Latency Denial-of-Service: Attacking the LLM Serving Framework, Not the Model](https://mlsanity.com/paper/2602.07878/) (2026)
- [Resource Consumption Threats in Large Language Models](https://mlsanity.com/paper/2603.16068/) (2026)

## Abstract (arXiv)

Object detection is a fundamental enabler for many real-time downstream applications such as autonomous driving, augmented reality and supply chain management. However, the algorithmic backbone of neural networks is brittle to imperceptible perturbations in the system inputs, which were generally known as misclassifying attacks. By targeting the real-time processing capability, a new class of latency attacks are reported recently. They exploit new attack surfaces in object detectors by creating a computational bottleneck in the post-processing module, that leads to cascading failure and puts the real-time downstream tasks at risks. In this work, we take an initial attempt to defend against this attack via background-attentive adversarial training that is also cognizant of the underlying hardware capabilities. We first draw system-level connections between latency attack and hardware capacity across heterogeneous GPU devices. Based on the particular adversarial behaviors, we utilize objectness loss as a proxy and build background attention into the adversarial training pipeline, and achieve a reasonable balance between clean and robust accuracy. The extensive experiments demonstrate the defense effectiveness of restoring real-time processing capability from 13 FPS to 43 FPS on Jetson Orin NX, with a better trade-off between the clean and robust accuracy.