# Runtime Monitoring and Fault Detection for Neural Network-Controlled Systems

arXiv: https://arxiv.org/abs/2403.16132 · HTML: https://mlsanity.com/paper/2403.16132/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 21 references that are in the map.
- First posted on arXiv 24 Mar 2024.

## Builds on (most cited first, 12 of 12)

- [Algorithms for Verifying Deep Neural Networks](https://mlsanity.com/paper/1903.06758/) (2019)
- [Activation Functions in Deep Learning: A Comprehensive Survey and Benchmark](https://mlsanity.com/paper/2109.14545/) (2021)
- [Branch and Bound for Piecewise Linear Neural Network Verification](https://mlsanity.com/paper/1909.06588/) (2019)
- [An approach to reachability analysis for feed-forward ReLU neural networks](https://mlsanity.com/paper/1706.07351/) (2017)
- [Reach-SDP: Reachability Analysis of Closed-Loop Systems with Neural Network Controllers via Semidefinite Programming](https://mlsanity.com/paper/2004.07876/) (2020)
- [Reachability Analysis of Neural Feedback Loops](https://mlsanity.com/paper/2108.04140/) (2021)
- [OVERT: An Algorithm for Safety Verification of Neural Network Control Policies for Nonlinear Systems](https://mlsanity.com/paper/2108.01220/) (2021)
- [Perception and Navigation in Autonomous Systems in the Era of Learning: A Survey](https://mlsanity.com/paper/2001.02319/) (2020)
- [Stability and feasibility of neural network-based controllers via output range analysis](https://mlsanity.com/paper/2004.00521/) (2020)
- [Safety Verification of Neural Feedback Systems Based on Constrained Zonotopes](https://mlsanity.com/paper/2204.00903/) (2022)
- [What, Indeed, is an Achievable Provable Guarantee for Learning-Enabled Safety Critical Systems](https://mlsanity.com/paper/2307.11784/) (2023)
- [Run-Time Safety Monitoring of Neural-Network-Enabled Dynamical Systems](https://mlsanity.com/paper/2101.08297/) (2021)

## Abstract (arXiv)

There is an emerging trend in applying deep learning methods to control complex nonlinear systems. This paper considers enhancing the runtime safety of nonlinear systems controlled by neural networks in the presence of disturbance and measurement noise. A robustly stable interval observer is designed to generate sound and precise lower and upper bounds for the neural network, nonlinear function, and system state. The obtained interval is utilised to monitor the real-time system safety and detect faults in the system outputs or actuators. An adaptive cruise control vehicular system is simulated to demonstrate effectiveness of the proposed design.