mlsanityData to 7 Oct 2026

Synthesize then Compare: Detecting Failures and Anomalies for Semantic Segmentation

arXiv 2003.08440 ↗First posted 18 Mar 2020cs.CVcited by 96

Not on the map. Why?

Abstract (arXiv)

The ability to detect failures and anomalies are fundamental requirements for building reliable systems for computer vision applications, especially safety-critical applications of semantic segmentation, such as autonomous driving and medical image analysis. In this paper, we systematically study failure and anomaly detection for semantic segmentation and propose a unified framework, consisting of two modules, to address these two related problems. The first module is an image synthesis module, which generates a synthesized image from a segmentation layout map, and the second is a comparison module, which computes the difference between the synthesized image and the input image. We validate our framework on three challenging datasets and improve the state-of-the-arts by large margins, i.e., 6% AUPR-Error on Cityscapes, 7% Pearson correlation on pancreatic tumor segmentation in MSD and 20% AUPR on StreetHazards anomaly segmentation.

Key facts. Cited by 96 papers in the map (data through 7 Oct 2026). First posted on arXiv 18 Mar 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 16; 2022: 23; 2023: 23; 2024: 18; 2025: 11; 2026: 5.

96 papers.

Sort:

By area:

  1. 2022A Unified Model for Multi-class Anomaly Detectioncited by 267
  2. 2021SegmentMeIfYouCan: A Benchmark for Anomaly Segmentationcited by 150
  3. 2021Pixel-wise Anomaly Detection in Complex Driving Scenescited by 109
  4. 2022CODA: A Real-World Road Corner Case Dataset for Object Detection in Autonomous Drivingcited by 100
  5. 2022GMMSeg: Gaussian Mixture based Generative Semantic Segmentation Modelscited by 87
  6. 2021Pixel-wise Energy-biased Abstention Learning for Anomaly Segmentation on Complex Urban Driving Scenescited by 83
  7. 2021Standardized Max Logits: A Simple yet Effective Approach for Identifying Unexpected Road Obstacles in Urban-Scene Segmentationcited by 77
  8. 2022RbA: Segmenting Unknown Regions Rejected by Allcited by 67
  9. 2025Impromptu VLA: Open Weights and Open Data for Driving Vision-Language-Action Modelscited by 60
  10. 2023FUTURE-AI: International consensus guideline for trustworthy and deployable artificial intelligence in healthcarecited by 59
  11. 2022Residual Pattern Learning for Pixel-wise Out-of-Distribution Detection in Semantic Segmentationcited by 55
  12. 2021Deep Metric Learning for Open World Semantic Segmentationcited by 51
38 more
  1. 2023Unmasking Anomalies in Road-Scene Segmentationcited by 45
  2. 2022DenseHybrid: Hybrid Anomaly Detection for Dense Open-set Recognitioncited by 42
  3. 2023On Advantages of Mask-level Recognition for Outlier-aware Segmentationcited by 42
  4. 2021Triggering Failures: Out-Of-Distribution detection by learning from local adversarial attacks in Semantic Segmentationcited by 40
  5. 2021Description of Corner Cases in Automated Driving: Goals and Challengescited by 35
  6. 2023Maskomaly:Zero-Shot Mask Anomaly Segmentationcited by 34
  7. 2022ADTR: Anomaly Detection Transformer with Feature Reconstructioncited by 31
  8. 2021Open-set 3D Object Detectioncited by 29
  9. 2022Open-world Semantic Segmentation for LIDAR Point Cloudscited by 29
  10. 2022Visual Anomaly Detection Via Partition Memory Bank Module and Error Estimationcited by 20
  11. 2022Two Video Data Sets for Tracking and Retrieval of Out of Distribution Objectscited by 19
  12. 2023Top-Down Visual Attention from Analysis by Synthesiscited by 19
  13. 2023Mask2Anomaly: Mask Transformer for Universal Open-set Segmentationcited by 18
  14. 2021Dense outlier detection and open-set recognition based on training with noisy negative imagescited by 17
  15. 2023Devil is in the Queries: Advancing Mask Transformers for Real-world Medical Image Segmentation and Out-of-Distribution Localizationcited by 16
  16. 2022Curved Geometric Networks for Visual Anomaly Recognitioncited by 15
  17. 2022Far Away in the Deep Space: Dense Nearest-Neighbor-Based Out-of-Distribution Detectioncited by 15
  18. 2023UGainS: Uncertainty Guided Anomaly Instance Segmentationcited by 15
  19. 2023Towards Training-free Open-world Segmentation via Image Prompt Foundation Modelscited by 15
  20. 2023ZePT: Zero-Shot Pan-Tumor Segmentation via Query-Disentangling and Self-Promptingcited by 15
  21. 2023CSL: Class-Agnostic Structure-Constrained Learning for Segmentation Including the Unseencited by 15
  22. 2023ATTA: Anomaly-aware Test-Time Adaptation for Out-of-Distribution Detection in Segmentationcited by 14
  23. 2024SAM-LAD: Segment Anything Model Meets Zero-Shot Logic Anomaly Detectioncited by 14
  24. 2021Detecting Anomalies in Semantic Segmentation with Prototypescited by 13
  25. 2023Open Scene Understanding: Grounded Situation Recognition Meets Segment Anything for Helping People with Visual Impairmentscited by 13
  26. 2023Panoptic Out-of-Distribution Segmentationcited by 13
  27. 2022Amodal Cityscapes: A New Dataset, its Generation, and an Amodal Semantic Segmentation Challenge Baselinecited by 12
  28. 2024Placing Objects in Context via Inpainting for Out-of-distribution Segmentationcited by 12
  29. 2024Comparative Benchmarking of Failure Detection Methods in Medical Image Segmentation: Unveiling the Role of Confidence Aggregationcited by 12
  30. 2024A Likelihood Ratio-Based Approach to Segmenting Unknown Objectscited by 12
  31. 2024Towards Understanding and Quantifying Uncertainty for Text-to-Image Generationcited by 11
  32. 2021Dense Out-of-Distribution Detection by Robust Learning on Synthetic Negative Datacited by 10
  33. 2023Survey on video anomaly detection in dynamic scenes with moving camerascited by 10
  34. 2021FSNet: A Failure Detection Framework for Semantic Segmentationcited by 9
  35. 2024Diffusion for Out-of-Distribution Detection on Road Scenes and Beyondcited by 9
  36. 2023Hybrid Open-set Segmentation with Synthetic Negative Datacited by 8
  37. 2023Concurrent Misclassification and Out-of-Distribution Detection for Semantic Segmentation via Energy-Based Normalizing Flowcited by 8
  38. 2024Latency-aware Road Anomaly Segmentation in Videos: A Photorealistic Dataset and New Metricscited by 8

50 most cited of 96 All 96 (JSON)