# Novelty Detection Through Model-Based Characterization of Neural Networks

arXiv: https://arxiv.org/abs/2008.06094 · HTML: https://mlsanity.com/paper/2008.06094/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Aug 2020.

## Built on by (most cited first, 17 of 17)

- [Introspective Learning : A Two-Stage Approach for Inference in Neural Networks](https://mlsanity.com/paper/2209.08425/) (2022)
- [Anomaly Detection via Self-organizing Map](https://mlsanity.com/paper/2107.09903/) (2021)
- [Contrastive Reasoning in Neural Networks](https://mlsanity.com/paper/2103.12329/) (2021)
- [Explanatory Paradigms in Neural Networks](https://mlsanity.com/paper/2202.11838/) (2022)
- [Gradient-based Novelty Detection Boosted by Self-supervised Binary Classification](https://mlsanity.com/paper/2112.09815/) (2021)
- [Gaussian Switch Sampling: A Second Order Approach to Active Learning](https://mlsanity.com/paper/2302.12018/) (2023)
- [Forgetful Active Learning with Switch Events: Efficient Sampling for Out-of-Distribution Data](https://mlsanity.com/paper/2301.05106/) (2023)
- [A Gating Model for Bias Calibration in Generalized Zero-shot Learning](https://mlsanity.com/paper/2203.04195/) (2022)
- [Example Forgetting: A Novel Approach to Explain and Interpret Deep Neural Networks in Seismic Interpretation](https://mlsanity.com/paper/2302.14644/) (2023)
- [Subspace Modeling for Fast Out-Of-Distribution and Anomaly Detection](https://mlsanity.com/paper/2203.10422/) (2022)
- [Outlier-Robust Group Inference via Gradient Space Clustering](https://mlsanity.com/paper/2210.06759/) (2022)
- [VOICE: Variance of Induced Contrastive Explanations to quantify Uncertainty in Neural Network Interpretability](https://mlsanity.com/paper/2406.00573/) (2024)
- [R2-AD2: Detecting Anomalies by Analysing the Raw Gradient](https://mlsanity.com/paper/2206.10259/) (2022)
- [Counterfactual Gradients-based Quantification of Prediction Trust in Neural Networks](https://mlsanity.com/paper/2405.13758/) (2024)
- [Energy-Based Anomaly Detection and Localization](https://mlsanity.com/paper/2105.03270/) (2021)
- [Explainable Machine Learning for Hydrocarbon Prospect Risking](https://mlsanity.com/paper/2212.07563/) (2022)
- [Learning Trajectory-Conditioned Relations to Predict Pedestrian Crossing Behavior](https://mlsanity.com/paper/2301.05796/) (2023)

## Abstract (arXiv)

In this paper, we propose a model-based characterization of neural networks to detect novel input types and conditions. Novelty detection is crucial to identify abnormal inputs that can significantly degrade the performance of machine learning algorithms. Majority of existing studies have focused on activation-based representations to detect abnormal inputs, which limits the characterization of abnormality from a data perspective. However, a model perspective can also be informative in terms of the novelties and abnormalities. To articulate the significance of the model perspective in novelty detection, we utilize backpropagated gradients. We conduct a comprehensive analysis to compare the representation capability of gradients with that of activation and show that the gradients outperform the activation in novel class and condition detection. We validate our approach using four image recognition datasets including MNIST, Fashion-MNIST, CIFAR-10, and CURE-TSR. We achieve a significant improvement on all four datasets with an average AUROC of 0.953, 0.918, 0.582, and 0.746, respectively.