mlsanityData to 7 Oct 2026

FSNet: A Failure Detection Framework for Semantic Segmentation

arXiv 2108.08748 ↗First posted 19 Aug 2021cs.CV cs.ROcited by 9

Not on the map. Why?

Abstract (arXiv)

Semantic segmentation is an important task that helps autonomous vehicles understand their surroundings and navigate safely. During deployment, even the most mature segmentation models are vulnerable to various external factors that can degrade the segmentation performance with potentially catastrophic consequences for the vehicle and its surroundings. To address this issue, we propose a failure detection framework to identify pixel-level misclassification. We do so by exploiting internal features of the segmentation model and training it simultaneously with a failure detection network. During deployment, the failure detector can flag areas in the image where the segmentation model have failed to segment correctly. We evaluate the proposed approach against state-of-the-art methods and achieve 12.30%, 9.46%, and 9.65% performance improvement in the AUPR-Error metric for Cityscapes, BDD100K, and Mapillary semantic segmentation datasets.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 24 of its 46 references that are in the map. First posted on arXiv 19 Aug 2021.

Builds on

24 of 46 references.

References · 24

  1. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,845
  2. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,427
  3. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
  4. 2016DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFscited by 3,239
  5. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,362
  6. 2017Rethinking Atrous Convolution for Semantic Image Segmentationcited by 2,154
  7. 2017Selective Classification for Deep Neural Networkscited by 799
  8. 2020Image Segmentation Using Deep Learning: A Surveycited by 659
  9. 2020A Unifying Review of Deep and Shallow Anomaly Detectioncited by 423
  10. 2019Deep Multi-modal Object Detection and Semantic Segmentation for Autonomous Driving: Datasets, Methods, and Challengescited by 313
  11. 2018Recent Advances in Open Set Recognition: A Surveycited by 309
  12. 2018To Trust Or Not To Trust A Classifiercited by 205
12 more
  1. 2019Addressing Failure Prediction by Learning Model Confidencecited by 183
  2. 20192017 Robotic Instrument Segmentation Challengecited by 166
  3. 2020Synthesize then Compare: Detecting Failures and Anomalies for Semantic Segmentationcited by 96
  4. 2018Automatic Instrument Segmentation in Robot-Assisted Surgery Using Deep Learningcited by 81
  5. 2018Leveraging Uncertainty Estimates for Predicting Segmentation Qualitycited by 44
  6. 2017Reverse Classification Accuracy: Predicting Segmentation Performance in the Absence of Ground Truthcited by 35
  7. 2021Run-Time Monitoring of Machine Learning for Robotic Perception: A Survey of Emerging Trendscited by 26
  8. 2018Real-time Prediction of Segmentation Qualitycited by 23
  9. 2018Convolutional CRFs for Semantic Segmentationcited by 18
  10. 2016Introspective Perception: Learning to Predict Failures in Vision Systemscited by 17
  11. 2019Small Obstacle Avoidance Based on RGB-D Semantic Segmentationcited by 13
  12. 2018Uncertainty-driven Sanity Check: Application to Postoperative Brain Tumor Cavity Segmentationcited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 2; 2023: 3; 2024: 1; 2025: 0; 2026: 2.

9 papers.

  1. 2021See Yourself in Others: Attending Multiple Tasks for Own Failure Detectioncited by 8
  2. 2022Monitoring of Perception Systems: Deterministic, Probabilistic, and Learning-based Fault Detection and Identificationcited by 6
  3. 2023Failure Detection for Motion Prediction of Autonomous Driving: An Uncertainty Perspectivecited by 4
  4. 2024Towards Ground-truth-free Evaluation of Any Segmentation in Medical Imagescited by 3
  5. 2023High-Quality Unknown Object Instance Segmentation via Quadruple Boundary Error Refinementcited by 1
  6. 2022CNN based Intelligent Streetlight Management Using Smart CCTV Camera and Semantic Segmentationcited by 0
  7. 2023SQA-SAM: Segmentation Quality Assessment for Medical Images Utilizing the Segment Anything Modelcited by 0
  8. 2026Generative Semantic Segmentation via an Observable Semantic-Image Interface and Hierarchical Generator Evidence Alignmentcited by 0
  9. 2026Beyond Uncertainty: Generalizable Failure Monitoring for Surgical Segmentation under Acquisition Degradationcited by 0