# Koopman-based generalization bound: New aspect for full-rank weights

arXiv: https://arxiv.org/abs/2302.05825 · HTML: https://mlsanity.com/paper/2302.05825/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 44 references that are in the map.
- First posted on arXiv 12 Feb 2023.

## Builds on (most cited first, 25 of 25)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [Spectrally-normalized margin bounds for neural networks](https://mlsanity.com/paper/1706.08498/) (2017)
- [Deep learning for universal linear embeddings of nonlinear dynamics](https://mlsanity.com/paper/1712.09707/) (2017)
- [A PAC-Bayesian Approach to Spectrally-Normalized Margin Bounds for Neural Networks](https://mlsanity.com/paper/1707.09564/) (2017)
- [Stronger generalization bounds for deep nets via a compression approach](https://mlsanity.com/paper/1802.05296/) (2018)
- [Norm-Based Capacity Control in Neural Networks](https://mlsanity.com/paper/1503.00036/) (2015)
- [Size-Independent Sample Complexity of Neural Networks](https://mlsanity.com/paper/1712.06541/) (2017)
- [Modern Koopman Theory for Dynamical Systems](https://mlsanity.com/paper/2102.12086/) (2021)
- [Sensitivity and Generalization in Neural Networks: an Empirical Study](https://mlsanity.com/paper/1802.08760/) (2018)
- [Orthogonal Convolutional Neural Networks](https://mlsanity.com/paper/1911.12207/) (2019)
- [Orthogonal Deep Neural Networks](https://mlsanity.com/paper/1905.05929/) (2019)
- [Data-dependent Sample Complexity of Deep Neural Networks via Lipschitz Augmentation](https://mlsanity.com/paper/1905.03684/) (2019)
- [Improved Sample Complexities for Deep Networks and Robust Classification via an All-Layer Margin](https://mlsanity.com/paper/1910.04284/) (2019)
- [Eigendecompositions of Transfer Operators in Reproducing Kernel Hilbert Spaces](https://mlsanity.com/paper/1712.01572/) (2017)
- [Compression based bound for non-compressed network: unified generalization error analysis of large compressible deep neural network](https://mlsanity.com/paper/1909.11274/) (2019)
- [Robust Fine-Tuning of Deep Neural Networks with Hessian-based Generalization Guarantees](https://mlsanity.com/paper/2206.02659/) (2022)
- [Optimizing Neural Networks via Koopman Operator Theory](https://mlsanity.com/paper/2006.02361/) (2020)
- [Truth or Backpropaganda? An Empirical Investigation of Deep Learning Theory](https://mlsanity.com/paper/1910.00359/) (2019)
- [An Operator Theoretic View on Pruning Deep Neural Networks](https://mlsanity.com/paper/2110.14856/) (2021)
- [Complex Unitary Recurrent Neural Networks using Scaled Cayley Transform](https://mlsanity.com/paper/1811.04142/) (2018)
- [Metric on Nonlinear Dynamical Systems with Perron-Frobenius Operators](https://mlsanity.com/paper/1805.12324/) (2018)
- [Krylov Subspace Method for Nonlinear Dynamical Systems with Random Noise](https://mlsanity.com/paper/1909.03634/) (2019)
- [Composition operators on reproducing kernel Hilbert spaces with analytic positive definite functions](https://mlsanity.com/paper/1911.11992/) (2019)
- [Stable Invariant Models via Koopman Spectra](https://mlsanity.com/paper/2207.07475/) (2022)

## Built on by (most cited first, 9 of 9)

- [Koopman operators with intrinsic observables in rigged reproducing kernel Hilbert spaces](https://mlsanity.com/paper/2403.02524/) (2024)
- [Why High-rank Neural Networks Generalize?: An Algebraic Framework with RKHSs](https://mlsanity.com/paper/2509.21895/) (2025)
- [Operator-Based Generalization Bound for Deep Learning: Insights on Multi-Task Learning](https://mlsanity.com/paper/2512.19184/) (2025)
- [On the Koopman-Based Generalization Bounds for Multi-Task Deep Learning](https://mlsanity.com/paper/2512.19199/) (2025)
- [Operator-Theoretic Generalization Bounds for Multitask Deep Learning](https://mlsanity.com/paper/2608.15982/) (2026)
- [Glocal Hypergradient Estimation with Koopman Operator](https://mlsanity.com/paper/2402.02741/) (2024)
- [Deep Koopman-layered Model with Universal Property Based on Toeplitz Matrices](https://mlsanity.com/paper/2410.02199/) (2024)
- [Unified generalization analysis for physics informed neural networks](https://mlsanity.com/paper/2605.13260/) (2026)
- [Sample-Weighted End-to-End Trace-Norm Geometry for Multitask Learning](https://mlsanity.com/paper/2609.29520/) (2026)

## Abstract (arXiv)

We propose a new bound for generalization of neural networks using Koopman operators. Whereas most of existing works focus on low-rank weight matrices, we focus on full-rank weight matrices. Our bound is tighter than existing norm-based bounds when the condition numbers of weight matrices are small. Especially, it is completely independent of the width of the network if the weight matrices are orthogonal. Our bound does not contradict to the existing bounds but is a complement to the existing bounds. As supported by several existing empirical results, low-rankness is not the only reason for generalization. Furthermore, our bound can be combined with the existing bounds to obtain a tighter bound. Our result sheds new light on understanding generalization of neural networks with full-rank weight matrices, and it provides a connection between operator-theoretic analysis and generalization of neural networks.