# Hardening RGB-D Object Recognition Systems against Adversarial Patch Attacks

arXiv: https://arxiv.org/abs/2309.07106 · HTML: https://mlsanity.com/paper/2309.07106/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 34 references that are in the map.
- First posted on arXiv 13 Sep 2023.

## Builds on (most cited first, 27 of 27)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [Similarity of Neural Network Representations Revisited](https://mlsanity.com/paper/1905.00414/) (2019)
- [ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness](https://mlsanity.com/paper/1811.12231/) (2018)
- [Evasion Attacks against Machine Learning at Test Time](https://mlsanity.com/paper/1708.06131/) (2017)
- [Adversarial Patch](https://mlsanity.com/paper/1712.09665/) (2017)
- [Wild Patterns: Ten Years After the Rise of Adversarial Machine Learning](https://mlsanity.com/paper/1712.03141/) (2017)
- [MagNet: a Two-Pronged Defense against Adversarial Examples](https://mlsanity.com/paper/1705.09064/) (2017)
- [Do Wide and Deep Networks Learn the Same Things? Uncovering How Neural Network Representations Vary with Width and Depth](https://mlsanity.com/paper/2010.15327/) (2020)
- [LaVAN: Localized and Visible Adversarial Noise](https://mlsanity.com/paper/1801.02608/) (2018)
- [Wild Patterns Reloaded: A Survey of Machine Learning Security against Training Data Poisoning](https://mlsanity.com/paper/2205.01992/) (2022)
- [EasyLabel: A Semi-Automatic Pixel-wise Object Annotation Tool for Creating Robotic RGB-D Datasets](https://mlsanity.com/paper/1902.01626/) (2019)
- [Multimodal Deep Learning for Robust RGB-D Object Recognition](https://mlsanity.com/paper/1507.06821/) (2015)
- [Exploring Adversarial Robustness of Multi-Sensor Perception Systems in Self Driving](https://mlsanity.com/paper/2101.06784/) (2021)
- [Deep Neural Rejection against Adversarial Examples](https://mlsanity.com/paper/1910.00470/) (2019)
- [Machine Learning Security against Data Poisoning: Are We There Yet?](https://mlsanity.com/paper/2204.05986/) (2022)
- [Is Deep Learning Safe for Robot Vision? Adversarial Examples against the iCub Humanoid](https://mlsanity.com/paper/1708.06939/) (2017)
- [A black-box adversarial attack for poisoning clustering](https://mlsanity.com/paper/2009.05474/) (2020)
- [Adversarial Attacks on Camera-LiDAR Models for 3D Car Detection](https://mlsanity.com/paper/2103.09448/) (2021)
- [On the Adversarial Robustness of Camera-based 3D Object Detection](https://mlsanity.com/paper/2301.10766/) (2023)
- [Investigating Vulnerability to Adversarial Examples on Multimodal Data Fusion in Deep Learning](https://mlsanity.com/paper/2005.10987/) (2020)
- [FADER: Fast Adversarial Example Rejection](https://mlsanity.com/paper/2010.09119/) (2020)
- [The Hammer and the Nut: Is Bilevel Optimization Really Needed to Poison Linear Classifiers?](https://mlsanity.com/paper/2103.12399/) (2021)
- [Recurrent Convolutional Fusion for RGB-D Object Recognition](https://mlsanity.com/paper/1806.01673/) (2018)
- [Adversarial Robustness of Deep Sensor Fusion Models](https://mlsanity.com/paper/2006.13192/) (2020)
- [Multimodal Deep Domain Adaptation](https://mlsanity.com/paper/1807.11697/) (2018)

## Built on by (most cited first, 2 of 2)

- [Evaluating the Evaluators: Trust in Adversarial Robustness Tests](https://mlsanity.com/paper/2507.03450/) (2025)
- [Rethinking Robustness in Machine Learning: A Posterior Agreement Approach](https://mlsanity.com/paper/2503.16271/) (2025)

## Abstract (arXiv)

RGB-D object recognition systems improve their predictive performances by fusing color and depth information, outperforming neural network architectures that rely solely on colors. While RGB-D systems are expected to be more robust to adversarial examples than RGB-only systems, they have also been proven to be highly vulnerable. Their robustness is similar even when the adversarial examples are generated by altering only the original images' colors. Different works highlighted the vulnerability of RGB-D systems; however, there is a lacking of technical explanations for this weakness. Hence, in our work, we bridge this gap by investigating the learned deep representation of RGB-D systems, discovering that color features make the function learned by the network more complex and, thus, more sensitive to small perturbations. To mitigate this problem, we propose a defense based on a detection mechanism that makes RGB-D systems more robust against adversarial examples. We empirically show that this defense improves the performances of RGB-D systems against adversarial examples even when they are computed ad-hoc to circumvent this detection mechanism, and that is also more effective than adversarial training.