# Fooling thermal infrared pedestrian detectors in real world using small bulbs

arXiv: https://arxiv.org/abs/2101.08154 · HTML: https://mlsanity.com/paper/2101.08154/

## Key facts

- Cited by 51 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 30 references that are in the map.
- First posted on arXiv 20 Jan 2021.

## Builds on (most cited first, 16 of 16)

- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [Towards Evaluating the Robustness of Neural Networks](https://mlsanity.com/paper/1608.04644/) (2016)
- [YOLOv3: An Incremental Improvement](https://mlsanity.com/paper/1804.02767/) (2018)
- [YOLO9000: Better, Faster, Stronger](https://mlsanity.com/paper/1612.08242/) (2016)
- [Cascade R-CNN: Delving into High Quality Object Detection](https://mlsanity.com/paper/1712.00726/) (2017)
- [Adversarial Machine Learning at Scale](https://mlsanity.com/paper/1611.01236/) (2016)
- [Universal adversarial perturbations](https://mlsanity.com/paper/1610.08401/) (2016)
- [Fooling automated surveillance cameras: adversarial patches to attack person detection](https://mlsanity.com/paper/1904.08653/) (2019)
- [Generating Adversarial Examples with Adversarial Networks](https://mlsanity.com/paper/1801.02610/) (2018)
- [Invisible Mask: Practical Attacks on Face Recognition with Infrared](https://mlsanity.com/paper/1803.04683/) (2018)
- [Adversarial T-shirt! Evading Person Detectors in A Physical World](https://mlsanity.com/paper/1910.11099/) (2019)
- [End-to-End Face Parsing via Interlinked Convolutional Neural Networks](https://mlsanity.com/paper/2002.04831/) (2020)
- [Linear Support Tensor Machine: Pedestrian Detection in Thermal Infrared Images](https://mlsanity.com/paper/1609.07878/) (2016)

## Built on by (most cited first, 30 of 51)

- [Adversarial Texture for Fooling Person Detectors in the Physical World](https://mlsanity.com/paper/2203.03373/) (2022)
- [Physical Adversarial Attack meets Computer Vision: A Decade Survey](https://mlsanity.com/paper/2209.15179/) (2022)
- [X-Adv: Physical Adversarial Object Attacks against X-ray Prohibited Item Detection](https://mlsanity.com/paper/2302.09491/) (2023)
- [Infrared Invisible Clothing:Hiding from Infrared Detectors at Multiple Angles in Real World](https://mlsanity.com/paper/2205.05909/) (2022)
- [Physically Realizable Natural-Looking Clothing Textures Evade Person Detectors via 3D Modeling](https://mlsanity.com/paper/2307.01778/) (2023)
- [Physically Adversarial Infrared Patches with Learnable Shapes and Locations](https://mlsanity.com/paper/2303.13868/) (2023)
- [HOTCOLD Block: Fooling Thermal Infrared Detectors with a Novel Wearable Design](https://mlsanity.com/paper/2212.05709/) (2022)
- [Unified Adversarial Patch for Cross-modal Attacks in the Physical World](https://mlsanity.com/paper/2307.07859/) (2023)
- [A Survey on Physical Adversarial Attack in Computer Vision](https://mlsanity.com/paper/2209.14262/) (2022)
- [Language-Driven Anchors for Zero-Shot Adversarial Robustness](https://mlsanity.com/paper/2301.13096/) (2023)
- [SoK: On the Semantic AI Security in Autonomous Driving](https://mlsanity.com/paper/2203.05314/) (2022)
- [SceneTAP: Scene-Coherent Typographic Adversarial Planner against Vision-Language Models in Real-World Environments](https://mlsanity.com/paper/2412.00114/) (2024)
- [Unified Adversarial Patch for Visible-Infrared Cross-modal Attacks in the Physical World](https://mlsanity.com/paper/2307.14682/) (2023)
- [Defensive Patches for Robust Recognition in the Physical World](https://mlsanity.com/paper/2204.06213/) (2022)
- [Physical Adversarial Attacks For Camera-based Smart Systems: Current Trends, Categorization, Applications, Research Challenges, and Future Outlook](https://mlsanity.com/paper/2308.06173/) (2023)
- [Recognizing Object by Components with Human Prior Knowledge Enhances Adversarial Robustness of Deep Neural Networks](https://mlsanity.com/paper/2212.01806/) (2022)
- [Physical Adversarial Attacks for Surveillance: A Survey](https://mlsanity.com/paper/2305.01074/) (2023)
- [On the Importance of Backbone to the Adversarial Robustness of Object Detectors](https://mlsanity.com/paper/2305.17438/) (2023)
- [Visual Adversarial Attacks and Defenses in the Physical World: A Survey](https://mlsanity.com/paper/2211.01671/) (2022)
- [Adversarial Infrared Blocks: A Multi-view Black-box Attack to Thermal Infrared Detectors in Physical World](https://mlsanity.com/paper/2304.10712/) (2023)
- [Adversarial Examples in the Physical World: A Survey](https://mlsanity.com/paper/2311.01473/) (2023)
- [Adversarial Infrared Curves: An Attack on Infrared Pedestrian Detectors in the Physical World](https://mlsanity.com/paper/2312.14217/) (2023)
- [State-of-the-art optical-based physical adversarial attacks for deep learning computer vision systems](https://mlsanity.com/paper/2303.12249/) (2023)
- [Infrared Adversarial Car Stickers](https://mlsanity.com/paper/2405.09924/) (2024)
- [CBA: Contextual Background Attack against Optical Aerial Detection in the Physical World](https://mlsanity.com/paper/2302.13519/) (2023)
- [Adversarial Neon Beam: A Light-based Physical Attack to DNNs](https://mlsanity.com/paper/2204.00853/) (2022)
- [Beyond Boundaries: A Comprehensive Survey of Transferable Attacks on AI Systems](https://mlsanity.com/paper/2311.11796/) (2023)
- [Physical Backdoor: Towards Temperature-based Backdoor Attacks in the Physical World](https://mlsanity.com/paper/2404.19417/) (2024)
- [A Comprehensive Study on the Robustness of Image Classification and Object Detection in Remote Sensing: Surveying and Benchmarking](https://mlsanity.com/paper/2306.12111/) (2023)
- [Real-world Adversarial Defense against Patch Attacks based on Diffusion Model](https://mlsanity.com/paper/2409.09406/) (2024)

## Abstract (arXiv)

Thermal infrared detection systems play an important role in many areas such as night security, autonomous driving, and body temperature detection. They have the unique advantages of passive imaging, temperature sensitivity and penetration. But the security of these systems themselves has not been fully explored, which poses risks in applying these systems. We propose a physical attack method with small bulbs on a board against the state of-the-art pedestrian detectors. Our goal is to make infrared pedestrian detectors unable to detect real-world pedestrians. Towards this goal, we first showed that it is possible to use two kinds of patches to attack the infrared pedestrian detector based on YOLOv3. The average precision (AP) dropped by 64.12% in the digital world, while a blank board with the same size caused the AP to drop by 29.69% only. After that, we designed and manufactured a physical board and successfully attacked YOLOv3 in the real world. In recorded videos, the physical board caused AP of the target detector to drop by 34.48%, while a blank board with the same size caused the AP to drop by 14.91% only. With the ensemble attack techniques, the designed physical board had good transferability to unseen detectors. We also proposed the first physical multispectral (infrared and visible) attack. By using a combination method, we successfully hide from the visible light and infrared object detection systems at the same time.