# Detection and Retrieval of Out-of-Distribution Objects in Semantic Segmentation

arXiv: https://arxiv.org/abs/2005.06831 · HTML: https://mlsanity.com/paper/2005.06831/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 May 2020.

## Built on by (most cited first, 20 of 20)

- [SegmentMeIfYouCan: A Benchmark for Anomaly Segmentation](https://mlsanity.com/paper/2104.14812/) (2021)
- [Pixel-wise Anomaly Detection in Complex Driving Scenes](https://mlsanity.com/paper/2103.05445/) (2021)
- [Triggering Failures: Out-Of-Distribution detection by learning from local adversarial attacks in Semantic Segmentation](https://mlsanity.com/paper/2108.01634/) (2021)
- [Inspect, Understand, Overcome: A Survey of Practical Methods for AI Safety](https://mlsanity.com/paper/2104.14235/) (2021)
- [Maskomaly:Zero-Shot Mask Anomaly Segmentation](https://mlsanity.com/paper/2305.16972/) (2023)
- [Two Video Data Sets for Tracking and Retrieval of Out of Distribution Objects](https://mlsanity.com/paper/2210.02074/) (2022)
- [Devil is in the Queries: Advancing Mask Transformers for Real-world Medical Image Segmentation and Out-of-Distribution Localization](https://mlsanity.com/paper/2304.00212/) (2023)
- [Hyperdimensional Feature Fusion for Out-Of-Distribution Detection](https://mlsanity.com/paper/2112.05341/) (2021)
- [Towards Unsupervised Open World Semantic Segmentation](https://mlsanity.com/paper/2201.01073/) (2022)
- [A Survey of Visual Sensory Anomaly Detection](https://mlsanity.com/paper/2202.07006/) (2022)
- [Have We Ever Encountered This Before? Retrieving Out-of-Distribution Road Obstacles from Driving Scenes](https://mlsanity.com/paper/2309.04302/) (2023)
- [Latency-aware Road Anomaly Segmentation in Videos: A Photorealistic Dataset and New Metrics](https://mlsanity.com/paper/2401.04942/) (2024)
- [Shifting Transformation Learning for Out-of-Distribution Detection](https://mlsanity.com/paper/2106.03899/) (2021)
- [Improving Anomaly Segmentation with Multi-Granularity Cross-Domain Alignment](https://mlsanity.com/paper/2308.08696/) (2023)
- [Evaluation of Out-of-Distribution Detection Performance on Autonomous Driving Datasets](https://mlsanity.com/paper/2401.17013/) (2024)
- [Detecting and Learning the Unknown in Semantic Segmentation](https://mlsanity.com/paper/2202.08700/) (2022)
- [AttEntropy: On the Generalization Ability of Supervised Semantic Segmentation Transformers to New Objects in New Domains](https://mlsanity.com/paper/2212.14397/) (2022)
- [Uncertainty and Prediction Quality Estimation for Semantic Segmentation via Graph Neural Networks](https://mlsanity.com/paper/2409.11373/) (2024)
- [Neural Network Meta Classifier: Improving the Reliability of Anomaly Segmentation](https://mlsanity.com/paper/2412.10765/) (2024)
- [Out-of-Distribution Object Detection in Street Scenes via Synthetic Outlier Exposure and Transfer Learning](https://mlsanity.com/paper/2603.16122/) (2026)

## Abstract (arXiv)

When deploying deep learning technology in self-driving cars, deep neural networks are constantly exposed to domain shifts. These include, e.g., changes in weather conditions, time of day, and long-term temporal shift. In this work we utilize a deep neural network trained on the Cityscapes dataset containing urban street scenes and infer images from a different dataset, the A2D2 dataset, containing also countryside and highway images. We present a novel pipeline for semantic segmenation that detects out-of-distribution (OOD) segments by means of the deep neural network's prediction and performs image retrieval after feature extraction and dimensionality reduction on image patches. In our experiments we demonstrate that the deployed OOD approach is suitable for detecting out-of-distribution concepts. Furthermore, we evaluate the image patch retrieval qualitatively as well as quantitatively by means of the semi-compatible A2D2 ground truth and obtain mAP values of up to 52.2%.