# Can I trust my anomaly detection system? A case study based on explainable AI

arXiv: https://arxiv.org/abs/2407.19951 · HTML: https://mlsanity.com/paper/2407.19951/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 23 references that are in the map.
- First posted on arXiv 29 Jul 2024.

## Builds on (most cited first, 12 of 12)

- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [Autoencoding beyond pixels using a learned similarity metric](https://mlsanity.com/paper/1512.09300/) (2015)
- [Deep Industrial Image Anomaly Detection: A Survey](https://mlsanity.com/paper/2301.11514/) (2023)
- [The Shapley Value in Machine Learning](https://mlsanity.com/paper/2202.05594/) (2022)
- [SHAP-IQ: Unified Approximation of any-order Shapley Interactions](https://mlsanity.com/paper/2303.01179/) (2023)
- [Explaining Anomalies Detected by Autoencoders Using SHAP](https://mlsanity.com/paper/1903.02407/) (2019)
- [A new interpretable unsupervised anomaly detection method based on residual explanation](https://mlsanity.com/paper/2103.07953/) (2021)
- [VAE-LIME: Deep Generative Model Based Approach for Local Data-Driven Model Interpretability Applied to the Ironmaking Industry](https://mlsanity.com/paper/2007.10256/) (2020)
- [Using Stratified Sampling to Improve LIME Image Explanations](https://mlsanity.com/paper/2403.17742/) (2024)

## Abstract (arXiv)

Generative models based on variational autoencoders are a popular technique for detecting anomalies in images in a semi-supervised context. A common approach employs the anomaly score to detect the presence of anomalies, and it is known to reach high level of accuracy on benchmark datasets. However, since anomaly scores are computed from reconstruction disparities, they often obscure the detection of various spurious features, raising concerns regarding their actual efficacy. This case study explores the robustness of an anomaly detection system based on variational autoencoder generative models through the use of eXplainable AI methods. The goal is to get a different perspective on the real performances of anomaly detectors that use reconstruction differences. In our case study we discovered that, in many cases, samples are detected as anomalous for the wrong or misleading factors.