mlsanityData to 14 Oct 2026

Multi-Scale Foreground-Background Confidence for Out-of-Distribution Segmentation

arXiv 2412.16990 ↗First posted 22 Dec 2024cs.CVcited by 3

Not on the map. Why?

Abstract (arXiv)

Deep neural networks have shown outstanding performance in computer vision tasks such as semantic segmentation and have defined the state-of-the-art. However, these segmentation models are trained on a closed and predefined set of semantic classes, which leads to significant prediction failures in open-world scenarios on unknown objects. As this behavior prevents the application in safety-critical applications such as automated driving, the detection and segmentation of these objects from outside their predefined semantic space (out-of-distribution (OOD) objects) is of the utmost importance. In this work, we present a multi-scale OOD segmentation method that exploits the confidence information of a foreground-background segmentation model. While semantic segmentation models are trained on specific classes, this restriction does not apply to foreground-background methods making them suitable for OOD segmentation. We consider the per pixel confidence score of the model prediction which is close to 1 for a pixel in a foreground object. By aggregating these confidence values for different sized patches, objects of various sizes can be identified in a single image. Our experiments show improved performance of our method in OOD segmentation compared to comparable baselines in the SegmentMeIfYouCan benchmark.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 40 of its 53 references that are in the map. First posted on arXiv 22 Dec 2024.

Builds on

40 of 53 references.

References · 40

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  3. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,448
  4. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,444
  5. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,926
  6. 2018Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentationcited by 3,223
  7. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,385
  8. 2018A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attackscited by 1,485
  9. 2022Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference timecited by 1,396
  10. 2017Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networkscited by 1,328
  11. 2022YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectorscited by 982
  12. 2023Universal Instance Perception as Object Discovery and Retrievalcited by 192
28 more
  1. 2023Cut and Learn for Unsupervised Object Detection and Instance Segmentationcited by 187
  2. 2021SegmentMeIfYouCan: A Benchmark for Anomaly Segmentationcited by 150
  3. 2018Evaluating Bayesian Deep Learning Methods for Semantic Segmentationcited by 143
  4. 2019Detecting the Unexpected via Image Resynthesiscited by 137
  5. 2020Entropy Maximization and Meta Classification for Out-Of-Distribution Detection in Semantic Segmentationcited by 133
  6. 2019The Fishyscapes Benchmark: Measuring Blind Spots in Semantic Segmentationcited by 122
  7. 2016Lost and Found: Detecting Small Road Hazards for Self-Driving Vehiclescited by 117
  8. 2021Pixel-wise Anomaly Detection in Complex Driving Scenescited by 109
  9. 2022PIDNet: A Real-time Semantic Segmentation Network Inspired by PID Controllerscited by 84
  10. 2021Pixel-wise Energy-biased Abstention Learning for Anomaly Segmentation on Complex Urban Driving Scenescited by 83
  11. 2022RbA: Segmenting Unknown Regions Rejected by Allcited by 67
  12. 2022Unsupervised Object Localization: Observing the Background to Discover Objectscited by 66
  13. 2022Residual Pattern Learning for Pixel-wise Out-of-Distribution Detection in Semantic Segmentationcited by 55
  14. 2023Unmasking Anomalies in Road-Scene Segmentationcited by 45
  15. 2023On Advantages of Mask-level Recognition for Outlier-aware Segmentationcited by 42
  16. 2021Triggering Failures: Out-Of-Distribution detection by learning from local adversarial attacks in Semantic Segmentationcited by 40
  17. 2023Maskomaly:Zero-Shot Mask Anomaly Segmentationcited by 34
  18. 2024Outlier detection by ensembling uncertainty with negative objectnesscited by 34
  19. 2020Detecting Road Obstacles by Erasing Themcited by 29
  20. 2024Open-World Semantic Segmentation Including Class Similaritycited by 24
  21. 2024PixOOD: Pixel-Level Out-of-Distribution Detectioncited by 20
  22. 2022Two Video Data Sets for Tracking and Retrieval of Out of Distribution Objectscited by 19
  23. 2022Far Away in the Deep Space: Dense Nearest-Neighbor-Based Out-of-Distribution Detectioncited by 15
  24. 2023ATTA: Anomaly-aware Test-Time Adaptation for Out-of-Distribution Detection in Segmentationcited by 14
  25. 2023Pixel-wise Gradient Uncertainty for Convolutional Neural Networks applied to Out-of-Distribution Segmentationcited by 11
  26. 2023Concurrent Misclassification and Out-of-Distribution Detection for Semantic Segmentation via Energy-Based Normalizing Flowcited by 8
  27. 2020Efficient Ensemble Model Generation for Uncertainty Estimation with Bayesian Approximation in Segmentationcited by 5
  28. 2022False Negative Reduction in Semantic Segmentation under Domain Shift using Depth Estimationcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 1.

3 papers.

  1. 2025Out-of-Distribution Segmentation in Autonomous Driving: Problems and State of the Artcited by 13
  2. 2025Out-of-Distribution Segmentation via Wasserstein-Based Evidential Uncertaintycited by 1
  3. 2026Combining Foundation Model Confidence and Monocular Depth for Training-Free Out-of-Distribution Segmentationcited by 0