mlsanityData to 14 Oct 2026

Stability and Performance Analysis of Discrete-Time ReLU Recurrent Neural Networks

arXiv 2405.05236 ↗First posted 8 May 2024eess.SY cs.LG cs.SYcited by 4

Not on the map. Why?

Abstract (arXiv)

This paper presents sufficient conditions for the stability and ℓ2-gain performance of recurrent neural networks (RNNs) with ReLU activation functions. These conditions are derived by combining Lyapunov/dissipativity theory with Quadratic Constraints (QCs) satisfied by repeated ReLUs. We write a general class of QCs for repeated RELUs using known properties for the scalar ReLU. Our stability and performance condition uses these QCs along with a "lifted" representation for the ReLU RNN. We show that the positive homogeneity property satisfied by a scalar ReLU does not expand the class of QCs for the repeated ReLU. We present examples to demonstrate the stability / performance condition and study the effect of the lifting horizon.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 12 of its 35 references that are in the map. First posted on arXiv 8 May 2024.

Builds on

12 of 35 references.

References · 12

  1. 2014Analysis and Design of Optimization Algorithms via Integral Quadratic Constraintscited by 105
  2. 2019Safety Verification and Robustness Analysis of Neural Networks via Quadratic Constraints and Semidefinite Programmingcited by 101
  3. 2020Stability Analysis using Quadratic Constraints for Systems with Neural Network Controllerscited by 44
  4. 2018Lyapunov Functions for First-Order Methods: Tight Automated Convergence Guaranteescited by 11
  5. 2023Robust Machine Learning Modeling for Predictive Control Using Lipschitz-Constrained Neural Networkscited by 8
  6. 2020l2 Induced Norm Analysis of Discrete-Time LTI Systems for Nonnegative Input Signals and Its Application to Stability Analysis of Recurrent Neural Networkscited by 5
  7. 2021Dissipativity and Integral Quadratic Constraints, Tailored computational robustness tests for complex interconnectionscited by 5
  8. 2018Convex searches for discrete-time Zames-Falb multiplierscited by 4
  9. 2021Reduced-Order Neural Network Synthesis with Robustness Guaranteescited by 3
  10. 2022Stability Analysis of Recurrent Neural Networks by IQC with Copositive Mutiplierscited by 3
  11. 2020Finite Step Performance of First-order Methods Using Interpolation Conditions Without Function Evaluationscited by 1
  12. 2022Absolute Stability via Lifting and Interpolationcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 2; 2026: 1.

4 papers.

  1. 2024A Complete Set of Quadratic Constraints for Repeated ReLU and Generalizationscited by 2
  2. 2025Local Stability and Region of Attraction Analysis for Neural Network Feedback Systems under Positivity Constraintscited by 2
  3. 2025Distributionally Robust Safety Verification of Neural Networks via Worst-Case CVaRcited by 0
  4. 2026Scalable Incremental Robustness Analysis of Neural Network Feedback Systemscited by 0