# Runtime Monitoring DNN-Based Perception

arXiv: https://arxiv.org/abs/2310.03999 · HTML: https://mlsanity.com/paper/2310.03999/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 28 references that are in the map.
- First posted on arXiv 6 Oct 2023.

## Builds on (most cited first, 16 of 16)

- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks](https://mlsanity.com/paper/1610.02136/) (2016)
- [A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks](https://mlsanity.com/paper/1807.03888/) (2018)
- [Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networks](https://mlsanity.com/paper/1706.02690/) (2017)
- [Outside the Box: Abstraction-Based Monitoring of Neural Networks](https://mlsanity.com/paper/1911.09032/) (2019)
- [Conformal Prediction for STL Runtime Verification](https://mlsanity.com/paper/2211.01539/) (2022)
- [Runtime Monitoring Neuron Activation Patterns](https://mlsanity.com/paper/1809.06573/) (2018)
- [RTAMT: Online Robustness Monitors from STL](https://mlsanity.com/paper/2005.11827/) (2020)
- [Detection of out-of-distribution samples using binary neuron activation patterns](https://mlsanity.com/paper/2212.14268/) (2022)
- [PerceMon: Online Monitoring for Perception Systems](https://mlsanity.com/paper/2108.08289/) (2021)
- [Dynamic Shielding for Reinforcement Learning in Black-Box Environments](https://mlsanity.com/paper/2207.13446/) (2022)
- [Towards Safety Verification of Direct Perception Neural Networks](https://mlsanity.com/paper/1904.04706/) (2019)
- [Provably-Robust Runtime Monitoring of Neuron Activation Patterns](https://mlsanity.com/paper/2011.11959/) (2020)
- [Hardware faults that matter: Understanding and Estimating the safety impact of hardware faults on object detection DNNs](https://mlsanity.com/paper/2209.03225/) (2022)
- [Customizable Reference Runtime Monitoring of Neural Networks using Resolution Boxes](https://mlsanity.com/paper/2104.14435/) (2021)
- [Monitoring Object Detection Abnormalities via Data-Label and Post-Algorithm Abstractions](https://mlsanity.com/paper/2103.15456/) (2021)

## Built on by (most cited first, 1 of 1)

- [Taming Silent Failures: A Framework for Verifiable AI Reliability](https://mlsanity.com/paper/2510.22224/) (2025)

## Abstract (arXiv)

Deep neural networks (DNNs) are instrumental in realizing complex perception systems. As many of these applications are safety-critical by design, engineering rigor is required to ensure that the functional insufficiency of the DNN-based perception is not the source of harm. In addition to conventional static verification and testing techniques employed during the design phase, there is a need for runtime verification techniques that can detect critical events, diagnose issues, and even enforce requirements. This tutorial aims to provide readers with a glimpse of techniques proposed in the literature. We start with classical methods proposed in the machine learning community, then highlight a few techniques proposed by the formal methods community. While we surely can observe similarities in the design of monitors, how the decision boundaries are created vary between the two communities. We conclude by highlighting the need to rigorously design monitors, where data availability outside the operational domain plays an important role.