# Convolutional Neural Networks as 2-D systems

arXiv: https://arxiv.org/abs/2303.03042 · HTML: https://mlsanity.com/paper/2303.03042/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 71 references that are in the map.
- First posted on arXiv 6 Mar 2023.

## Builds on (most cited first, 30 of 31)

- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [Lipschitz regularity of deep neural networks: analysis and efficient estimation](https://mlsanity.com/paper/1805.10965/) (2018)
- [Efficient and Accurate Estimation of Lipschitz Constants for Deep Neural Networks](https://mlsanity.com/paper/1906.04893/) (2019)
- [Inductive Biases for Deep Learning of Higher-Level Cognition](https://mlsanity.com/paper/2011.15091/) (2020)
- [Sorting out Lipschitz function approximation](https://mlsanity.com/paper/1811.05381/) (2018)
- [Semidefinite relaxations for certifying robustness to adversarial examples](https://mlsanity.com/paper/1811.01057/) (2018)
- [On the Inductive Bias of Neural Tangent Kernels](https://mlsanity.com/paper/1905.12173/) (2019)
- [Orthogonal Convolutional Neural Networks](https://mlsanity.com/paper/1911.12207/) (2019)
- [Safety Verification and Robustness Analysis of Neural Networks via Quadratic Constraints and Semidefinite Programming](https://mlsanity.com/paper/1903.01287/) (2019)
- [Training robust neural networks using Lipschitz bounds](https://mlsanity.com/paper/2005.02929/) (2020)
- [Pooling Methods in Deep Neural Networks, a Review](https://mlsanity.com/paper/2009.07485/) (2020)
- [Lipschitz Bounded Equilibrium Networks](https://mlsanity.com/paper/2010.01732/) (2020)
- [Recurrent Equilibrium Networks: Flexible Dynamic Models with Guaranteed Stability and Robustness](https://mlsanity.com/paper/2104.05942/) (2021)
- [Stability Analysis using Quadratic Constraints for Systems with Neural Network Controllers](https://mlsanity.com/paper/2006.07579/) (2020)
- [Efficient-CapsNet: Capsule Network with Self-Attention Routing](https://mlsanity.com/paper/2101.12491/) (2021)
- [Towards Certifying L-infinity Robustness using Neural Networks with L-inf-dist Neurons](https://mlsanity.com/paper/2102.05363/) (2021)
- [Imitation Learning with Stability and Safety Guarantees](https://mlsanity.com/paper/2012.09293/) (2020)
- [Pay attention to your loss: understanding misconceptions about 1-Lipschitz neural networks](https://mlsanity.com/paper/2104.05097/) (2021)
- [Deep neural networks, generic universal interpolation, and controlled ODEs](https://mlsanity.com/paper/1908.07838/) (2019)
- [Approximating Lipschitz continuous functions with GroupSort neural networks](https://mlsanity.com/paper/2006.05254/) (2020)
- [Certifying Incremental Quadratic Constraints for Neural Networks via Convex Optimization](https://mlsanity.com/paper/2012.05981/) (2020)
- [Norm-preserving Orthogonal Permutation Linear Unit Activation Functions (OPLU)](https://mlsanity.com/paper/1604.02313/) (2016)
- [Semialgebraic Representation of Monotone Deep Equilibrium Models and Applications to Certification](https://mlsanity.com/paper/2106.01453/) (2021)
- [Chordal Sparsity for Lipschitz Constant Estimation of Deep Neural Networks](https://mlsanity.com/paper/2204.00846/) (2022)
- [Lipschitz constant estimation for 1D convolutional neural networks](https://mlsanity.com/paper/2211.15253/) (2022)
- [Offset-free setpoint tracking using neural network controllers](https://mlsanity.com/paper/2011.14006/) (2020)
- [Large Norms of CNN Layers Do Not Hurt Adversarial Robustness](https://mlsanity.com/paper/2009.08435/) (2020)

## Built on by (most cited first, 5 of 5)

- [Lipschitz constant estimation for general neural network architectures using control tools](https://mlsanity.com/paper/2405.01125/) (2024)
- [Novel Quadratic Constraints for Extending LipSDP beyond Slope-Restricted Activations](https://mlsanity.com/paper/2401.14033/) (2024)
- [State space representations of the Roesser type for convolutional layers](https://mlsanity.com/paper/2403.11938/) (2024)
- [LipKernel: Lipschitz-Bounded Convolutional Neural Networks via Dissipative Layers](https://mlsanity.com/paper/2410.22258/) (2024)
- [Convex training of Lipschitz-regularized shallow neural networks](https://mlsanity.com/paper/2606.19652/) (2026)

## Abstract (arXiv)

This paper introduces a novel representation of convolutional Neural Networks (CNNs) in terms of 2-D dynamical systems. To this end, the usual description of convolutional layers with convolution kernels, i.e., the impulse responses of linear filters, is realized in state space as a linear time-invariant 2-D system. The overall convolutional Neural Network composed of convolutional layers and nonlinear activation functions is then viewed as a 2-D version of a Lur'e system, i.e., a linear dynamical system interconnected with static nonlinear components. One benefit of this 2-D Lur'e system perspective on CNNs is that we can use robust control theory much more efficiently for Lipschitz constant estimation than previously possible.