# Adversarial Manhole: Challenging Monocular Depth Estimation and Semantic Segmentation Models with Patch Attack

arXiv: https://arxiv.org/abs/2408.14879 · HTML: https://mlsanity.com/paper/2408.14879/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 15 of its 17 references that are in the map.
- First posted on arXiv 27 Aug 2024.

## Builds on (most cited first, 15 of 15)

- [Pyramid Scene Parsing Network](https://mlsanity.com/paper/1612.01105/) (2016)
- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [Understanding Deep Image Representations by Inverting Them](https://mlsanity.com/paper/1412.0035/) (2014)
- [ICNet for Real-Time Semantic Segmentation on High-Resolution Images](https://mlsanity.com/paper/1704.08545/) (2017)
- [The Temporal Opportunist: Self-Supervised Multi-Frame Monocular Depth](https://mlsanity.com/paper/2104.14540/) (2021)
- [Self-Supervised Monocular Depth Hints](https://mlsanity.com/paper/1909.09051/) (2019)
- [On the Robustness of Semantic Segmentation Models to Adversarial Attacks](https://mlsanity.com/paper/1711.09856/) (2017)
- [Physical Attack on Monocular Depth Estimation with Optimal Adversarial Patches](https://mlsanity.com/paper/2207.04718/) (2022)
- [ACTIVE: Towards Highly Transferable 3D Physical Camouflage for Universal and Robust Vehicle Evasion](https://mlsanity.com/paper/2308.07009/) (2023)
- [Adversarial Patch Attacks on Monocular Depth Estimation Networks](https://mlsanity.com/paper/2010.03072/) (2020)
- [Survey on Unsupervised Domain Adaptation for Semantic Segmentation for Visual Perception in Automated Driving](https://mlsanity.com/paper/2304.11928/) (2023)
- [A Survey on Physical Adversarial Attack in Computer Vision](https://mlsanity.com/paper/2209.14262/) (2022)
- [Certified Defences Against Adversarial Patch Attacks on Semantic Segmentation](https://mlsanity.com/paper/2209.05980/) (2022)
- [SAAM: Stealthy Adversarial Attack on Monocular Depth Estimation](https://mlsanity.com/paper/2308.03108/) (2023)
- [APARATE: Adaptive Adversarial Patch for CNN-based Monocular Depth Estimation for Autonomous Navigation](https://mlsanity.com/paper/2303.01351/) (2023)

## Abstract (arXiv)

Monocular depth estimation (MDE) and semantic segmentation (SS) are crucial for the navigation and environmental interpretation of many autonomous driving systems. However, their vulnerability to practical adversarial attacks is a significant concern. This paper presents a novel adversarial attack using practical patches that mimic manhole covers to deceive MDE and SS models. The goal is to cause these systems to misinterpret scenes, leading to false detections of near obstacles or non-passable objects. We use Depth Planar Mapping to precisely position these patches on road surfaces, enhancing the attack's effectiveness. Our experiments show that these adversarial patches cause a 43% relative error in MDE and achieve a 96% attack success rate in SS. These patches create affected error regions over twice their size in MDE and approximately equal to their size in SS. Our studies also confirm the patch's effectiveness in physical simulations, the adaptability of the patches across different target models, and the effectiveness of our proposed modules, highlighting their practical implications.