Data to 7 Oct 2026

The Fishyscapes Benchmark: Measuring Blind Spots in Semantic Segmentation

arXiv 1904.03215 ↗First posted 5 Apr 2019cs.CVcited by 122

Not on the map. Why?

Abstract (arXiv)

Deep learning has enabled impressive progress in the accuracy of semantic segmentation. Yet, the ability to estimate uncertainty and detect failure is key for safety-critical applications like autonomous driving. Existing uncertainty estimates have mostly been evaluated on simple tasks, and it is unclear whether these methods generalize to more complex scenarios. We present Fishyscapes, the first public benchmark for uncertainty estimation in a real-world task of semantic segmentation for urban driving. It evaluates pixel-wise uncertainty estimates towards the detection of anomalous objects in front of the vehicle. We~adapt state-of-the-art methods to recent semantic segmentation models and compare approaches based on softmax confidence, Bayesian learning, and embedding density. Our results show that anomaly detection is far from solved even for ordinary situations, while our benchmark allows measuring advancements beyond the state-of-the-art.

Key facts. Cited by 122 papers in the map (data through 7 Oct 2026). First posted on arXiv 5 Apr 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 16; 2022: 20; 2023: 21; 2024: 27; 2025: 27; 2026: 11.

122 papers.

Sort:

By area:

2022VOS: Learning What You Don't Know by Virtual Outlier Synthesiscited by 300
2021OpenGAN: Open-Set Recognition via Open Data Generationcited by 135
2021Pixel-wise Anomaly Detection in Complex Driving Scenescited by 109
2022CODA: A Real-World Road Corner Case Dataset for Object Detection in Autonomous Drivingcited by 100
2022GMMSeg: Gaussian Mixture based Generative Semantic Segmentation Modelscited by 87
2021Pixel-wise Energy-biased Abstention Learning for Anomaly Segmentation on Complex Urban Driving Scenescited by 83
2021Standardized Max Logits: A Simple yet Effective Approach for Identifying Unexpected Road Obstacles in Urban-Scene Segmentationcited by 77
2022RbA: Segmenting Unknown Regions Rejected by Allcited by 67
2024A Survey on Autonomous Driving Datasets: Statistics, Annotation Quality, and a Future Outlookcited by 65
2021On the Practicality of Deterministic Epistemic Uncertaintycited by 62
2021Semantics for Robotic Mapping, Perception and Interaction: A Surveycited by 56
2022Residual Pattern Learning for Pixel-wise Out-of-Distribution Detection in Semantic Segmentationcited by 55

38 more

2023Unmasking Anomalies in Road-Scene Segmentationcited by 45
2022DenseHybrid: Hybrid Anomaly Detection for Dense Open-set Recognitioncited by 42
2023On Advantages of Mask-level Recognition for Outlier-aware Segmentationcited by 42
2021Triggering Failures: Out-Of-Distribution detection by learning from local adversarial attacks in Semantic Segmentationcited by 40
2023Maskomaly:Zero-Shot Mask Anomaly Segmentationcited by 34
2024Outlier detection by ensembling uncertainty with negative objectnesscited by 34
2021An Application-Driven Conceptualization of Corner Cases for Perception in Highly Automated Drivingcited by 33
2024MultiOOD: Scaling Out-of-Distribution Detection for Multiple Modalitiescited by 28
2023Reliability in Semantic Segmentation: Can We Use Synthetic Data?cited by 24
2024Open-World Semantic Segmentation Including Class Similaritycited by 22
2024PixOOD: Pixel-Level Out-of-Distribution Detectioncited by 20
2022Two Video Data Sets for Tracking and Retrieval of Out of Distribution Objectscited by 19
2023Mask2Anomaly: Mask Transformer for Universal Open-set Segmentationcited by 18
2023Perception Datasets for Anomaly Detection in Autonomous Driving: A Surveycited by 16
2023Balanced Energy Regularization Loss for Out-of-distribution Detectioncited by 16
2022Far Away in the Deep Space: Dense Nearest-Neighbor-Based Out-of-Distribution Detectioncited by 15
2023UGainS: Uncertainty Guided Anomaly Instance Segmentationcited by 15
2024Learning Feature Inversion for Multi-class Anomaly Detection under General-purpose COCO-AD Benchmarkcited by 15
2023ATTA: Anomaly-aware Test-Time Adaptation for Out-of-Distribution Detection in Segmentationcited by 14
2024AnoVox: A Benchmark for Multimodal Anomaly Detection in Autonomous Drivingcited by 14
2025Object Detection with Multimodal Large Vision-Language Models: An In-depth Reviewcited by 14
2021Hyperdimensional Feature Fusion for Out-Of-Distribution Detectioncited by 13
2024Looking 3D: Anomaly Detection with 2D-3D Alignmentcited by 13
2025Out-of-Distribution Segmentation in Autonomous Driving: Problems and State of the Artcited by 13
2024A Likelihood Ratio-Based Approach to Segmenting Unknown Objectscited by 12
2022Ad-datasets: a meta-collection of data sets for autonomous drivingcited by 11
2023Pixel-wise Gradient Uncertainty for Convolutional Neural Networks applied to Out-of-Distribution Segmentationcited by 11
2021Dense Out-of-Distribution Detection by Robust Learning on Synthetic Negative Datacited by 10
2022Visual Recognition by Requestcited by 10
2025Spotting the Unexpected (STU): A 3D LiDAR Dataset for Anomaly Segmentation in Autonomous Drivingcited by 10
2021A Quantitative Comparison of Epistemic Uncertainty Maps Applied to Multi-Class Segmentationcited by 9
2022Segmenting Known Objects and Unseen Unknowns without Prior Knowledgecited by 9
2024Diffusion for Out-of-Distribution Detection on Road Scenes and Beyondcited by 9
2024OpenAD: Open-World Autonomous Driving Benchmark for 3D Object Detectioncited by 9
2021Learning Uncertainty For Safety-Oriented Semantic Segmentation In Autonomous Drivingcited by 8
2022Dual Decision Improves Open-Set Panoptic Segmentationcited by 8
2023Hybrid Open-set Segmentation with Synthetic Negative Datacited by 8
2024OoDIS: Anomaly Instance Segmentation and Detection Benchmarkcited by 8

50 most cited of 122 All 122 (JSON)