# Into the Unknown: Active Monitoring of Neural Networks

arXiv: https://arxiv.org/abs/2009.06429 · HTML: https://mlsanity.com/paper/2009.06429/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Sep 2020.

## Built on by (most cited first, 19 of 19)

- [An Abstraction-Refinement Approach to Verifying Convolutional Neural Networks](https://mlsanity.com/paper/2201.01978/) (2022)
- [Verification-Aided Deep Ensemble Selection](https://mlsanity.com/paper/2202.03898/) (2022)
- [Correct-by-Construction Runtime Enforcement in AI -- A Survey](https://mlsanity.com/paper/2208.14426/) (2022)
- [Verifying Generalization in Deep Learning](https://mlsanity.com/paper/2302.05745/) (2023)
- [Unifying Evaluation of Machine Learning Safety Monitors](https://mlsanity.com/paper/2208.14660/) (2022)
- [Runtime Monitoring for Out-of-Distribution Detection in Object Detection Neural Networks](https://mlsanity.com/paper/2212.07773/) (2022)
- [Customizable Reference Runtime Monitoring of Neural Networks using Resolution Boxes](https://mlsanity.com/paper/2104.14435/) (2021)
- [SpecRepair: Counter-Example Guided Safety Repair of Deep Neural Networks](https://mlsanity.com/paper/2106.01917/) (2021)
- [What, Indeed, is an Achievable Provable Guarantee for Learning-Enabled Safety Critical Systems](https://mlsanity.com/paper/2307.11784/) (2023)
- [Online Safety Analysis for LLMs: a Benchmark, an Assessment, and a Path Forward](https://mlsanity.com/paper/2404.08517/) (2024)
- [Learning Minimal Neural Specifications](https://mlsanity.com/paper/2404.04662/) (2024)
- [Safety Monitoring of Machine Learning Perception Functions: a Survey](https://mlsanity.com/paper/2412.06869/) (2024)
- [Gaussian-Based and Outside-the-Box Runtime Monitoring Join Forces](https://mlsanity.com/paper/2410.06051/) (2024)
- [Edge Case Detection in Automated Driving: Methods, Challenges, and Future Directions](https://mlsanity.com/paper/2410.08491/) (2024)
- [Runtime Safety Monitoring of Deep Neural Networks for Perception: A Survey](https://mlsanity.com/paper/2511.05982/) (2025)
- [Abstraction and Symbolic Execution of Deep Neural Networks with Bayesian Approximation of Hidden Features](https://mlsanity.com/paper/2103.03704/) (2021)
- [Negotiated Representations to Prevent Forgetting in Machine Learning Applications](https://mlsanity.com/paper/2312.00237/) (2023)
- [DeepProv: Behavioral Characterization and Repair of Neural Networks via Inference Provenance Graph Analysis](https://mlsanity.com/paper/2509.26562/) (2025)
- [Mining Verdict Boundaries for Neural Network Verification](https://mlsanity.com/paper/2607.28954/) (2026)

## Abstract (arXiv)

Neural-network classifiers achieve high accuracy when predicting the class of an input that they were trained to identify. Maintaining this accuracy in dynamic environments, where inputs frequently fall outside the fixed set of initially known classes, remains a challenge. The typical approach is to detect inputs from novel classes and retrain the classifier on an augmented dataset. However, not only the classifier but also the detection mechanism needs to adapt in order to distinguish between newly learned and yet unknown input classes. To address this challenge, we introduce an algorithmic framework for active monitoring of a neural network. A monitor wrapped in our framework operates in parallel with the neural network and interacts with a human user via a series of interpretable labeling queries for incremental adaptation. In addition, we propose an adaptive quantitative monitor to improve precision. An experimental evaluation on a diverse set of benchmarks with varying numbers of classes confirms the benefits of our active monitoring framework in dynamic scenarios.