# Extraction of nonlinearity in neural networks with Koopman operator

arXiv: https://arxiv.org/abs/2402.11740 · HTML: https://mlsanity.com/paper/2402.11740/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 30 references that are in the map.
- First posted on arXiv 18 Feb 2024.

## Builds on (most cited first, 9 of 9)

- [Solving high-dimensional partial differential equations using deep learning](https://mlsanity.com/paper/1707.02568/) (2017)
- [Deep learning for universal linear embeddings of nonlinear dynamics](https://mlsanity.com/paper/1712.09707/) (2017)
- [Pruning and Quantization for Deep Neural Network Acceleration: A Survey](https://mlsanity.com/paper/2101.09671/) (2021)
- [Deep Neural Networks Motivated by Partial Differential Equations](https://mlsanity.com/paper/1804.04272/) (2018)
- [Linear predictors for nonlinear dynamical systems: Koopman operator meets model predictive control](https://mlsanity.com/paper/1611.03537/) (2016)
- [Machine learning in spectral domain](https://mlsanity.com/paper/2005.14436/) (2020)
- [Koopman operator learning using invertible neural networks](https://mlsanity.com/paper/2306.17396/) (2023)
- [Tensor-based computation of metastable and coherent sets](https://mlsanity.com/paper/1908.04741/) (2019)
- [Stable Invariant Models via Koopman Spectra](https://mlsanity.com/paper/2207.07475/) (2022)

## Built on by (most cited first, 3 of 3)

- [Representing Neural Network Layers as Linear Operations via Koopman Operator Theory](https://mlsanity.com/paper/2409.01308/) (2024)
- [Extraction of linearized models from pre-trained networks via knowledge distillation](https://mlsanity.com/paper/2604.06732/) (2026)
- [Dynamic Mode Decomposition along Depth in Vision Transformers](https://mlsanity.com/paper/2605.07556/) (2026)

## Abstract (arXiv)

Nonlinearity plays a crucial role in deep neural networks. In this paper, we investigate the degree to which the nonlinearity of the neural network is essential. For this purpose, we employ the Koopman operator, extended dynamic mode decomposition, and the tensor-train format. The Koopman operator approach has been recently developed in physics and nonlinear sciences; the Koopman operator deals with the time evolution in the observable space instead of the state space. Since we can replace the nonlinearity in the state space with the linearity in the observable space, it is a hopeful candidate for understanding complex behavior in nonlinear systems. Here, we analyze learned neural networks for the classification problems. As a result, the replacement of the nonlinear middle layers with the Koopman matrix yields enough accuracy in numerical experiments. In addition, we confirm that the pruning of the Koopman matrix gives sufficient accuracy even at high compression ratios. These results indicate the possibility of extracting some features in the neural networks with the Koopman operator approach.