# This is not what I imagined: Error Detection for Semantic Segmentation through Visual Dissimilarity

arXiv: https://arxiv.org/abs/1909.00676 · HTML: https://mlsanity.com/paper/1909.00676/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Sep 2019.

## Built on by (most cited first, 16 of 16)

- [Pixel-wise Anomaly Detection in Complex Driving Scenes](https://mlsanity.com/paper/2103.05445/) (2021)
- [Pixel-wise Energy-biased Abstention Learning for Anomaly Segmentation on Complex Urban Driving Scenes](https://mlsanity.com/paper/2111.12264/) (2021)
- [RbA: Segmenting Unknown Regions Rejected by All](https://mlsanity.com/paper/2211.14293/) (2022)
- [CSL: Class-Agnostic Structure-Constrained Learning for Segmentation Including the Unseen](https://mlsanity.com/paper/2312.05538/) (2023)
- [Placing Objects in Context via Inpainting for Out-of-distribution Segmentation](https://mlsanity.com/paper/2402.16392/) (2024)
- [A Likelihood Ratio-Based Approach to Segmenting Unknown Objects](https://mlsanity.com/paper/2409.06424/) (2024)
- [See Yourself in Others: Attending Multiple Tasks for Own Failure Detection](https://mlsanity.com/paper/2110.02549/) (2021)
- [Improving Anomaly Segmentation with Multi-Granularity Cross-Domain Alignment](https://mlsanity.com/paper/2308.08696/) (2023)
- [Consensus Synergizes with Memory: A Simple Approach for Anomaly Segmentation in Urban Scenes](https://mlsanity.com/paper/2111.15463/) (2021)
- [Failure Detection for Motion Prediction of Autonomous Driving: An Uncertainty Perspective](https://mlsanity.com/paper/2301.04421/) (2023)
- [Panoramic Out-of-Distribution Segmentation](https://mlsanity.com/paper/2505.03539/) (2025)
- [Out-of-Distribution Semantic Occupancy Prediction](https://mlsanity.com/paper/2506.21185/) (2025)
- [Experiments on Anomaly Detection in Autonomous Driving by Forward-Backward Style Transfers](https://mlsanity.com/paper/2207.06055/) (2022)
- [From Pixel to Mask: A Survey of Out-of-Distribution Segmentation](https://mlsanity.com/paper/2508.10309/) (2025)
- [Leveraging Text-Driven Semantic Variation for Robust OOD Segmentation](https://mlsanity.com/paper/2511.07238/) (2025)
- [ProOOD: Prototype-Guided Out-of-Distribution 3D Occupancy Prediction](https://mlsanity.com/paper/2604.01081/) (2026)

## Abstract (arXiv)

There has been a remarkable progress in the accuracy of semantic segmentation due to the capabilities of deep learning. Unfortunately, these methods are not able to generalize much further than the distribution of their training data and fail to handle out-of-distribution classes appropriately. This limits the applicability to autonomous or safety critical systems. We propose a novel method leveraging generative models to detect wrongly segmented or out-of-distribution instances. Conditioned on the predicted semantic segmentation, an RGB image is generated. We then learn a dissimilarity metric that compares the generated image with the original input and detects inconsistencies introduced by the semantic segmentation. We present test cases for outlier and misclassification detection and evaluate our method qualitatively and quantitatively on multiple datasets.