# Stability and Performance Analysis of Discrete-Time ReLU Recurrent Neural Networks

arXiv: https://arxiv.org/abs/2405.05236 · HTML: https://mlsanity.com/paper/2405.05236/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 35 references that are in the map.
- First posted on arXiv 8 May 2024.

## Builds on (most cited first, 12 of 12)

- [Analysis and Design of Optimization Algorithms via Integral Quadratic Constraints](https://mlsanity.com/paper/1408.3595/) (2014)
- [Safety Verification and Robustness Analysis of Neural Networks via Quadratic Constraints and Semidefinite Programming](https://mlsanity.com/paper/1903.01287/) (2019)
- [Stability Analysis using Quadratic Constraints for Systems with Neural Network Controllers](https://mlsanity.com/paper/2006.07579/) (2020)
- [Lyapunov Functions for First-Order Methods: Tight Automated Convergence Guarantees](https://mlsanity.com/paper/1803.06073/) (2018)
- [Robust Machine Learning Modeling for Predictive Control Using Lipschitz-Constrained Neural Networks](https://mlsanity.com/paper/2308.13721/) (2023)
- [l2 Induced Norm Analysis of Discrete-Time LTI Systems for Nonnegative Input Signals and Its Application to Stability Analysis of Recurrent Neural Networks](https://mlsanity.com/paper/2011.12726/) (2020)
- [Dissipativity and Integral Quadratic Constraints, Tailored computational robustness tests for complex interconnections](https://mlsanity.com/paper/2105.07401/) (2021)
- [Convex searches for discrete-time Zames-Falb multipliers](https://mlsanity.com/paper/1812.02397/) (2018)
- [Reduced-Order Neural Network Synthesis with Robustness Guarantees](https://mlsanity.com/paper/2102.09284/) (2021)
- [Stability Analysis of Recurrent Neural Networks by IQC with Copositive Mutipliers](https://mlsanity.com/paper/2202.04592/) (2022)
- [Finite Step Performance of First-order Methods Using Interpolation Conditions Without Function Evaluations](https://mlsanity.com/paper/2005.01825/) (2020)
- [Absolute Stability via Lifting and Interpolation](https://mlsanity.com/paper/2209.06412/) (2022)

## Built on by (most cited first, 4 of 4)

- [A Complete Set of Quadratic Constraints for Repeated ReLU and Generalizations](https://mlsanity.com/paper/2407.06888/) (2024)
- [Local Stability and Region of Attraction Analysis for Neural Network Feedback Systems under Positivity Constraints](https://mlsanity.com/paper/2505.22889/) (2025)
- [Distributionally Robust Safety Verification of Neural Networks via Worst-Case CVaR](https://mlsanity.com/paper/2509.17413/) (2025)
- [Scalable Incremental Robustness Analysis of Neural Network Feedback Systems](https://mlsanity.com/paper/2609.22576/) (2026)

## Abstract (arXiv)

This paper presents sufficient conditions for the stability and ℓ2-gain performance of recurrent neural networks (RNNs) with ReLU activation functions. These conditions are derived by combining Lyapunov/dissipativity theory with Quadratic Constraints (QCs) satisfied by repeated ReLUs. We write a general class of QCs for repeated RELUs using known properties for the scalar ReLU. Our stability and performance condition uses these QCs along with a "lifted" representation for the ReLU RNN. We show that the positive homogeneity property satisfied by a scalar ReLU does not expand the class of QCs for the repeated ReLU. We present examples to demonstrate the stability / performance condition and study the effect of the lifting horizon.