mlsanityData to 14 Oct 2026

Detecting the Unexpected via Image Resynthesis

arXiv 1904.07595 ↗First posted 16 Apr 2019cs.CVcited by 137

Not on the map. Why?

Abstract (arXiv)

Classical semantic segmentation methods, including the recent deep learning ones, assume that all classes observed at test time have been seen during training. In this paper, we tackle the more realistic scenario where unexpected objects of unknown classes can appear at test time. The main trends in this area either leverage the notion of prediction uncertainty to flag the regions with low confidence as unknown, or rely on autoencoders and highlight poorly-decoded regions. Having observed that, in both cases, the detected regions typically do not correspond to unexpected objects, in this paper, we introduce a drastically different strategy: It relies on the intuition that the network will produce spurious labels in regions depicting unexpected objects. Therefore, resynthesizing the image from the resulting semantic map will yield significant appearance differences with respect to the input image. In other words, we translate the problem of detecting unknown classes to one of identifying poorly-resynthesized image regions. We show that this outperforms both uncertainty- and autoencoder-based methods.

Key facts. Cited by 137 papers in the map (data through 14 Oct 2026). First posted on arXiv 16 Apr 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 17; 2022: 25; 2023: 28; 2024: 29; 2025: 27; 2026: 11.

137 papers.

Sort:

By area:

  1. 2023EfficientAD: Accurate Visual Anomaly Detection at Millisecond-Level Latenciescited by 160
  2. 2021SegmentMeIfYouCan: A Benchmark for Anomaly Segmentationcited by 150
  3. 2021Pixel-wise Anomaly Detection in Complex Driving Scenescited by 109
  4. 2022CODA: A Real-World Road Corner Case Dataset for Object Detection in Autonomous Drivingcited by 100
  5. 2022GMMSeg: Gaussian Mixture based Generative Semantic Segmentation Modelscited by 87
  6. 2025Cosmos-Drive-Dreams: Scalable Synthetic Driving Data Generation with World Foundation Modelscited by 84
  7. 2021Pixel-wise Energy-biased Abstention Learning for Anomaly Segmentation on Complex Urban Driving Scenescited by 83
  8. 2021Standardized Max Logits: A Simple yet Effective Approach for Identifying Unexpected Road Obstacles in Urban-Scene Segmentationcited by 77
  9. 2022RbA: Segmenting Unknown Regions Rejected by Allcited by 67
  10. 2024A Survey on Autonomous Driving Datasets: Statistics, Annotation Quality, and a Future Outlookcited by 66
  11. 2025Impromptu VLA: Open Weights and Open Data for Driving Vision-Language-Action Modelscited by 60
  12. 2022Residual Pattern Learning for Pixel-wise Out-of-Distribution Detection in Semantic Segmentationcited by 55
38 more
  1. 2021Deep Metric Learning for Open World Semantic Segmentationcited by 51
  2. 2023Unmasking Anomalies in Road-Scene Segmentationcited by 45
  3. 2022DenseHybrid: Hybrid Anomaly Detection for Dense Open-set Recognitioncited by 42
  4. 2023On Advantages of Mask-level Recognition for Outlier-aware Segmentationcited by 42
  5. 2021Triggering Failures: Out-Of-Distribution detection by learning from local adversarial attacks in Semantic Segmentationcited by 40
  6. 2023Maskomaly:Zero-Shot Mask Anomaly Segmentationcited by 34
  7. 2024Outlier detection by ensembling uncertainty with negative objectnesscited by 34
  8. 2021An Application-Driven Conceptualization of Corner Cases for Perception in Highly Automated Drivingcited by 33
  9. 2021Open-set 3D Object Detectioncited by 29
  10. 2022Open-world Semantic Segmentation for LIDAR Point Cloudscited by 29
  11. 2021Corner Cases for Visual Perception in Automated Driving: Some Guidance on Detection Approachescited by 27
  12. 2024Out-of-Distribution Detection: A Task-Oriented Survey of Recent Advancescited by 26
  13. 2023Segment Every Out-of-Distribution Objectcited by 21
  14. 2024PixOOD: Pixel-Level Out-of-Distribution Detectioncited by 20
  15. 2022Two Video Data Sets for Tracking and Retrieval of Out of Distribution Objectscited by 19
  16. 2023LaRS: A Diverse Panoptic Maritime Obstacle Detection Dataset and Benchmarkcited by 19
  17. 2024MirrorCheck: Efficient Adversarial Defense for Vision-Language Modelscited by 19
  18. 2023Mask2Anomaly: Mask Transformer for Universal Open-set Segmentationcited by 18
  19. 2023Perception Datasets for Anomaly Detection in Autonomous Driving: A Surveycited by 16
  20. 2023Devil is in the Queries: Advancing Mask Transformers for Real-world Medical Image Segmentation and Out-of-Distribution Localizationcited by 16
  21. 2023Balanced Energy Regularization Loss for Out-of-distribution Detectioncited by 16
  22. 2022Curved Geometric Networks for Visual Anomaly Recognitioncited by 15
  23. 2022Far Away in the Deep Space: Dense Nearest-Neighbor-Based Out-of-Distribution Detectioncited by 15
  24. 2023UGainS: Uncertainty Guided Anomaly Instance Segmentationcited by 15
  25. 2023CSL: Class-Agnostic Structure-Constrained Learning for Segmentation Including the Unseencited by 15
  26. 2023ATTA: Anomaly-aware Test-Time Adaptation for Out-of-Distribution Detection in Segmentationcited by 14
  27. 2021Detecting Anomalies in Semantic Segmentation with Prototypescited by 13
  28. 2022Towards Unsupervised Open World Semantic Segmentationcited by 13
  29. 2024COFT-AD: COntrastive Fine-Tuning for Few-Shot Anomaly Detectioncited by 13
  30. 2025Out-of-Distribution Segmentation in Autonomous Driving: Problems and State of the Artcited by 13
  31. 2024Placing Objects in Context via Inpainting for Out-of-distribution Segmentationcited by 12
  32. 2024A Likelihood Ratio-Based Approach to Segmenting Unknown Objectscited by 12
  33. 2022A Survey of Visual Sensory Anomaly Detectioncited by 11
  34. 2023Pixel-wise Gradient Uncertainty for Convolutional Neural Networks applied to Out-of-Distribution Segmentationcited by 11
  35. 2024Towards Understanding and Quantifying Uncertainty for Text-to-Image Generationcited by 11
  36. 2021Dense Out-of-Distribution Detection by Robust Learning on Synthetic Negative Datacited by 10
  37. 2023Survey on video anomaly detection in dynamic scenes with moving camerascited by 10
  38. 2024Diffusion for Out-of-Distribution Detection on Road Scenes and Beyondcited by 9

50 most cited of 137 All 137 (JSON)