# False Negative Reduction in Semantic Segmentation under Domain Shift using Depth Estimation

arXiv: https://arxiv.org/abs/2207.03513 · HTML: https://mlsanity.com/paper/2207.03513/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 51 references that are in the map.
- First posted on arXiv 7 Jul 2022.

## Builds on (most cited first, 30 of 33)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1802.02611/) (2018)
- [Deep High-Resolution Representation Learning for Visual Recognition](https://mlsanity.com/paper/1908.07919/) (2019)
- [Panoptic Segmentation](https://mlsanity.com/paper/1801.00868/) (2018)
- [From Big to Small: Multi-Scale Local Planar Guidance for Monocular Depth Estimation](https://mlsanity.com/paper/1907.10326/) (2019)
- [RobustNet: Improving Domain Generalization in Urban-Scene Segmentation via Instance Selective Whitening](https://mlsanity.com/paper/2103.15597/) (2021)
- [A2D2: Audi Autonomous Driving Dataset](https://mlsanity.com/paper/2004.06320/) (2020)
- [Wider or Deeper: Revisiting the ResNet Model for Visual Recognition](https://mlsanity.com/paper/1611.10080/) (2016)
- [IDD: A Dataset for Exploring Problems of Autonomous Navigation in Unconstrained Environments](https://mlsanity.com/paper/1811.10200/) (2018)
- [Lost and Found: Detecting Small Road Hazards for Self-Driving Vehicles](https://mlsanity.com/paper/1609.04653/) (2016)
- [Pixel Contrastive-Consistent Semi-Supervised Semantic Segmentation](https://mlsanity.com/paper/2108.09025/) (2021)
- [Style-Hallucinated Dual Consistency Learning for Domain Generalized Semantic Segmentation](https://mlsanity.com/paper/2204.02548/) (2022)
- [Domain Adaptive Semantic Segmentation with Self-Supervised Depth Estimation](https://mlsanity.com/paper/2104.13613/) (2021)
- [WildNet: Learning Domain Generalized Semantic Segmentation from the Wild](https://mlsanity.com/paper/2204.01446/) (2022)
- [Three Ways to Improve Semantic Segmentation with Self-Supervised Depth Estimation](https://mlsanity.com/paper/2012.10782/) (2020)
- [Dual Graph Convolutional Network for Semantic Segmentation](https://mlsanity.com/paper/1909.06121/) (2019)
- [Prediction Error Meta Classification in Semantic Segmentation: Detection via Aggregated Dispersion Measures of Softmax Probabilities](https://mlsanity.com/paper/1811.00648/) (2018)
- [Automated Synthetic-to-Real Generalization](https://mlsanity.com/paper/2007.06965/) (2020)
- [The Domain Shift Problem of Medical Image Segmentation and Vendor-Adaptation by Unet-GAN](https://mlsanity.com/paper/1910.13681/) (2019)
- [WEDGE: Web-Image Assisted Domain Generalization for Semantic Segmentation](https://mlsanity.com/paper/2109.14196/) (2021)
- [Time-Dynamic Estimates of the Reliability of Deep Semantic Segmentation Networks](https://mlsanity.com/paper/1911.05075/) (2019)
- [Striking the Right Balance: Recall Loss for Semantic Segmentation](https://mlsanity.com/paper/2106.14917/) (2021)
- [Improving Video Instance Segmentation by Light-weight Temporal Uncertainty Estimates](https://mlsanity.com/paper/2012.07504/) (2020)
- [Plugging Self-Supervised Monocular Depth into Unsupervised Domain Adaptation for Semantic Segmentation](https://mlsanity.com/paper/2110.06685/) (2021)
- [Exploiting Depth from Single Monocular Images for Object Detection and Semantic Segmentation](https://mlsanity.com/paper/1610.01706/) (2016)
- [MetaFusion: Controlled False-Negative Reduction of Minority Classes in Semantic Segmentation](https://mlsanity.com/paper/1912.07420/) (2019)
- [False Negative Reduction in Video Instance Segmentation using Uncertainty Estimates](https://mlsanity.com/paper/2106.14474/) (2021)
- [Importance-Aware Semantic Segmentation with Efficient Pyramidal Context Network for Navigational Assistant Systems](https://mlsanity.com/paper/1907.11066/) (2019)

## Built on by (most cited first, 3 of 3)

- [Uncertainty-based Detection of Adversarial Attacks in Semantic Segmentation](https://mlsanity.com/paper/2305.12825/) (2023)
- [Multi-Scale Foreground-Background Confidence for Out-of-Distribution Segmentation](https://mlsanity.com/paper/2412.16990/) (2024)
- [Uncertainty and Prediction Quality Estimation for Semantic Segmentation via Graph Neural Networks](https://mlsanity.com/paper/2409.11373/) (2024)

## Abstract (arXiv)

State-of-the-art deep neural networks demonstrate outstanding performance in semantic segmentation. However, their performance is tied to the domain represented by the training data. Open world scenarios cause inaccurate predictions which is hazardous in safety relevant applications like automated driving. In this work, we enhance semantic segmentation predictions using monocular depth estimation to improve segmentation by reducing the occurrence of non-detected objects in presence of domain shift. To this end, we infer a depth heatmap via a modified segmentation network which generates foreground-background masks, operating in parallel to a given semantic segmentation network. Both segmentation masks are aggregated with a focus on foreground classes (here road users) to reduce false negatives. To also reduce the occurrence of false positives, we apply a pruning based on uncertainty estimates. Our approach is modular in a sense that it post-processes the output of any semantic segmentation network. In our experiments, we observe less non-detected objects of most important classes and an enhanced generalization to other domains compared to the basic semantic segmentation prediction.