# Maskomaly:Zero-Shot Mask Anomaly Segmentation

arXiv: https://arxiv.org/abs/2305.16972 · HTML: https://mlsanity.com/paper/2305.16972/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- Builds on 41 of its 52 references that are in the map.
- First posted on arXiv 26 May 2023.

## Builds on (most cited first, 30 of 41)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1802.02611/) (2018)
- [SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers](https://mlsanity.com/paper/2105.15203/) (2021)
- [Pyramid Scene Parsing Network](https://mlsanity.com/paper/1612.01105/) (2016)
- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks](https://mlsanity.com/paper/1610.02136/) (2016)
- [Masked-attention Mask Transformer for Universal Image Segmentation](https://mlsanity.com/paper/2112.01527/) (2021)
- [Rethinking Atrous Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1706.05587/) (2017)
- [Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networks](https://mlsanity.com/paper/1706.02690/) (2017)
- [Energy-based Out-of-distribution Detection](https://mlsanity.com/paper/2010.03759/) (2020)
- [Per-Pixel Classification is Not All You Need for Semantic Segmentation](https://mlsanity.com/paper/2107.06278/) (2021)
- [Deep Anomaly Detection with Outlier Exposure](https://mlsanity.com/paper/1812.04606/) (2018)
- [Scaling Out-of-Distribution Detection for Real-World Settings](https://mlsanity.com/paper/1911.11132/) (2019)
- [Do Deep Generative Models Know What They Don't Know?](https://mlsanity.com/paper/1810.09136/) (2018)
- [Training Confidence-calibrated Classifiers for Detecting Out-of-Distribution Samples](https://mlsanity.com/paper/1711.09325/) (2017)
- [Input complexity and out-of-distribution detection with likelihood-based generative models](https://mlsanity.com/paper/1909.11480/) (2019)
- [SegmentMeIfYouCan: A Benchmark for Anomaly Segmentation](https://mlsanity.com/paper/2104.14812/) (2021)
- [Detecting the Unexpected via Image Resynthesis](https://mlsanity.com/paper/1904.07595/) (2019)
- [Improving Semantic Segmentation via Video Propagation and Label Relaxation](https://mlsanity.com/paper/1812.01593/) (2018)
- [Entropy Maximization and Meta Classification for Out-Of-Distribution Detection in Semantic Segmentation](https://mlsanity.com/paper/2012.06575/) (2020)
- [The Fishyscapes Benchmark: Measuring Blind Spots in Semantic Segmentation](https://mlsanity.com/paper/1904.03215/) (2019)
- [Pixel-wise Anomaly Detection in Complex Driving Scenes](https://mlsanity.com/paper/2103.05445/) (2021)
- [Is Out-of-Distribution Detection Learnable?](https://mlsanity.com/paper/2210.14707/) (2022)
- [Synthesize then Compare: Detecting Failures and Anomalies for Semantic Segmentation](https://mlsanity.com/paper/2003.08440/) (2020)
- [Understanding Failures in Out-of-Distribution Detection with Deep Generative Models](https://mlsanity.com/paper/2107.06908/) (2021)
- [GMMSeg: Gaussian Mixture based Generative Semantic Segmentation Models](https://mlsanity.com/paper/2210.02025/) (2022)
- [Pixel-wise Energy-biased Abstention Learning for Anomaly Segmentation on Complex Urban Driving Scenes](https://mlsanity.com/paper/2111.12264/) (2021)

## Built on by (most cited first, 30 of 34)

- [Outlier detection by ensembling uncertainty with negative objectness](https://mlsanity.com/paper/2402.15374/) (2024)
- [Open-World Semantic Segmentation Including Class Similarity](https://mlsanity.com/paper/2403.07532/) (2024)
- [Target before Shooting: Accurate Anomaly Detection and Localization under One Millisecond via Cascade Patch Retrieval](https://mlsanity.com/paper/2308.06748/) (2023)
- [Mask2Anomaly: Mask Transformer for Universal Open-set Segmentation](https://mlsanity.com/paper/2309.04573/) (2023)
- [AnoVox: A Benchmark for Multimodal Anomaly Detection in Autonomous Driving](https://mlsanity.com/paper/2405.07865/) (2024)
- [Out-of-Distribution Segmentation in Autonomous Driving: Problems and State of the Art](https://mlsanity.com/paper/2503.08695/) (2025)
- [A Likelihood Ratio-Based Approach to Segmenting Unknown Objects](https://mlsanity.com/paper/2409.06424/) (2024)
- [CoralVOS: Dataset and Benchmark for Coral Video Segmentation](https://mlsanity.com/paper/2310.01946/) (2023)
- [Diffusion for Out-of-Distribution Detection on Road Scenes and Beyond](https://mlsanity.com/paper/2407.15739/) (2024)
- [Prior2Former -- Evidential Modeling of Mask Transformers for Assumption-Free Open-World Panoptic Segmentation](https://mlsanity.com/paper/2504.04841/) (2025)
- [Finding Dino: A Plug-and-Play Framework for Zero-Shot Detection of Out-of-Distribution Objects Using Prototypes](https://mlsanity.com/paper/2404.07664/) (2024)
- [MC-PanDA: Mask Confidence for Panoptic Domain Adaptation](https://mlsanity.com/paper/2407.14110/) (2024)
- [VL4AD: Vision-Language Models Improve Pixel-wise Anomaly Detection](https://mlsanity.com/paper/2409.17330/) (2024)
- [UMAD: Unsupervised Mask-Level Anomaly Detection for Autonomous Driving](https://mlsanity.com/paper/2406.06370/) (2024)
- [Segment-Level Road Obstacle Detection Using Visual Foundation Model Priors and Likelihood Ratios](https://mlsanity.com/paper/2412.05707/) (2024)
- [Multi-Scale Foreground-Background Confidence for Out-of-Distribution Segmentation](https://mlsanity.com/paper/2412.16990/) (2024)
- [Uncertainty-Aware Likelihood Ratio Estimation for Pixel-Wise Out-of-Distribution Detection](https://mlsanity.com/paper/2508.00587/) (2025)
- [Open-World Panoptic Segmentation](https://mlsanity.com/paper/2412.12740/) (2024)
- [Panoramic Out-of-Distribution Segmentation](https://mlsanity.com/paper/2505.03539/) (2025)
- [Anomalies by Synthesis: Anomaly Detection using Generative Diffusion Models for Off-Road Navigation](https://mlsanity.com/paper/2505.22805/) (2025)
- [Out-of-Distribution Semantic Occupancy Prediction](https://mlsanity.com/paper/2506.21185/) (2025)
- [Road Obstacle Video Segmentation](https://mlsanity.com/paper/2509.13181/) (2025)
- [Random Walk on Pixel Manifolds for Anomaly Segmentation of Complex Driving Scenes](https://mlsanity.com/paper/2404.17961/) (2024)
- [Neural Network Meta Classifier: Improving the Reliability of Anomaly Segmentation](https://mlsanity.com/paper/2412.10765/) (2024)
- [Multi-modality Anomaly Segmentation on the Road](https://mlsanity.com/paper/2503.17712/) (2025)
- [Objectomaly: Objectness-Aware Refinement for OoD Segmentation with Structural Consistency and Boundary Precision](https://mlsanity.com/paper/2507.07460/) (2025)
- [Training-Free Out-Of-Distribution Segmentation With Foundation Models](https://mlsanity.com/paper/2510.02909/) (2025)
- [Language-Guided Open-World Anomaly Segmentation](https://mlsanity.com/paper/2512.01427/) (2025)
- [Vision-Language Feature Alignment for Road Anomaly Segmentation](https://mlsanity.com/paper/2603.01029/) (2026)
- [Road-Aware Anomaly Segmentation with Query-Guided Polygons and CLIP in Autonomous Driving](https://mlsanity.com/paper/2607.04304/) (2026)

## Abstract (arXiv)

We present a simple and practical framework for anomaly segmentation called Maskomaly. It builds upon mask-based standard semantic segmentation networks by adding a simple inference-time post-processing step which leverages the raw mask outputs of such networks. Maskomaly does not require additional training and only adds a small computational overhead to inference. Most importantly, it does not require anomalous data at training. We show top results for our method on SMIYC, RoadAnomaly, and StreetHazards. On the most central benchmark, SMIYC, Maskomaly outperforms all directly comparable approaches. Further, we introduce a novel metric that benefits the development of robust anomaly segmentation methods and demonstrate its informativeness on RoadAnomaly.