# Hack The Box: Fooling Deep Learning Abstraction-Based Monitors

arXiv: https://arxiv.org/abs/2107.04764 · HTML: https://mlsanity.com/paper/2107.04764/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 9 of its 35 references that are in the map.
- First posted on arXiv 10 Jul 2021.

## Builds on (most cited first, 9 of 9)

- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Adversarial Attacks on Neural Networks for Graph Data](https://mlsanity.com/paper/1805.07984/) (2018)
- [Generative Probabilistic Novelty Detection with Adversarial Autoencoders](https://mlsanity.com/paper/1807.02588/) (2018)
- [Anomaly Detection using One-Class Neural Networks](https://mlsanity.com/paper/1802.06360/) (2018)
- [Outside the Box: Abstraction-Based Monitoring of Neural Networks](https://mlsanity.com/paper/1911.09032/) (2019)
- [Distance-based Confidence Score for Neural Network Classifiers](https://mlsanity.com/paper/1709.09844/) (2017)
- [Analyzing Adversarial Attacks Against Deep Learning for Intrusion Detection in IoT Networks](https://mlsanity.com/paper/1905.05137/) (2019)
- [Launching Adversarial Attacks against Network Intrusion Detection Systems for IoT](https://mlsanity.com/paper/2104.12426/) (2021)
- [Bayesian Anomaly Detection and Classification](https://mlsanity.com/paper/1902.08627/) (2019)

## Abstract (arXiv)

Deep learning is a type of machine learning that adapts a deep hierarchy of concepts. Deep learning classifiers link the most basic version of concepts at the input layer to the most abstract version of concepts at the output layer, also known as a class or label. However, once trained over a finite set of classes, some deep learning models do not have the power to say that a given input does not belong to any of the classes and simply cannot be linked. Correctly invalidating the prediction of unrelated classes is a challenging problem that has been tackled in many ways in the literature. Novelty detection gives deep learning the ability to output "do not know" for novel/unseen classes. Still, no attention has been given to the security aspects of novelty detection. In this paper, we consider the case study of abstraction-based novelty detection and show that it is not robust against adversarial samples. Moreover, we show the feasibility of crafting adversarial samples that fool the deep learning classifier and bypass the novelty detection monitoring at the same time. In other words, these monitoring boxes are hackable. We demonstrate that novelty detection itself ends up as an attack surface.