mlsanityData to 14 Oct 2026

Koopman-based generalization bound: New aspect for full-rank weights

arXiv 2302.05825 ↗First posted 12 Feb 2023cs.LG math.FA stat.MLcited by 9

Not on the map. Why?

Abstract (arXiv)

We propose a new bound for generalization of neural networks using Koopman operators. Whereas most of existing works focus on low-rank weight matrices, we focus on full-rank weight matrices. Our bound is tighter than existing norm-based bounds when the condition numbers of weight matrices are small. Especially, it is completely independent of the width of the network if the weight matrices are orthogonal. Our bound does not contradict to the existing bounds but is a complement to the existing bounds. As supported by several existing empirical results, low-rankness is not the only reason for generalization. Furthermore, our bound can be combined with the existing bounds to obtain a tighter bound. Our result sheds new light on understanding generalization of neural networks with full-rank weight matrices, and it provides a connection between operator-theoretic analysis and generalization of neural networks.

Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Builds on 25 of its 44 references that are in the map. First posted on arXiv 12 Feb 2023.

Builds on

25 of 44 references.

References · 25

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,611
  3. 2017Spectrally-normalized margin bounds for neural networkscited by 686
  4. 2017Deep learning for universal linear embeddings of nonlinear dynamicscited by 494
  5. 2017A PAC-Bayesian Approach to Spectrally-Normalized Margin Bounds for Neural Networkscited by 314
  6. 2018Stronger generalization bounds for deep nets via a compression approachcited by 307
  7. 2015Norm-Based Capacity Control in Neural Networkscited by 299
  8. 2017Size-Independent Sample Complexity of Neural Networkscited by 290
  9. 2021Modern Koopman Theory for Dynamical Systemscited by 276
  10. 2018Sensitivity and Generalization in Neural Networks: an Empirical Studycited by 189
  11. 2019Orthogonal Convolutional Neural Networkscited by 123
  12. 2019Orthogonal Deep Neural Networkscited by 72
13 more
  1. 2019Data-dependent Sample Complexity of Deep Neural Networks via Lipschitz Augmentationcited by 71
  2. 2019Improved Sample Complexities for Deep Networks and Robust Classification via an All-Layer Margincited by 56
  3. 2017Eigendecompositions of Transfer Operators in Reproducing Kernel Hilbert Spacescited by 48
  4. 2019Compression based bound for non-compressed network: unified generalization error analysis of large compressible deep neural networkcited by 39
  5. 2022Robust Fine-Tuning of Deep Neural Networks with Hessian-based Generalization Guaranteescited by 34
  6. 2020Optimizing Neural Networks via Koopman Operator Theorycited by 33
  7. 2019Truth or Backpropaganda? An Empirical Investigation of Deep Learning Theorycited by 21
  8. 2021An Operator Theoretic View on Pruning Deep Neural Networkscited by 18
  9. 2018Complex Unitary Recurrent Neural Networks using Scaled Cayley Transformcited by 12
  10. 2018Metric on Nonlinear Dynamical Systems with Perron-Frobenius Operatorscited by 7
  11. 2019Krylov Subspace Method for Nonlinear Dynamical Systems with Random Noisecited by 7
  12. 2019Composition operators on reproducing kernel Hilbert spaces with analytic positive definite functionscited by 6
  13. 2022Stable Invariant Models via Koopman Spectracited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 3; 2025: 3; 2026: 3.

9 papers.

  1. 2024Koopman operators with intrinsic observables in rigged reproducing kernel Hilbert spacescited by 9
  2. 2025Why High-rank Neural Networks Generalize?: An Algebraic Framework with RKHSscited by 1
  3. 2025Operator-Based Generalization Bound for Deep Learning: Insights on Multi-Task Learningcited by 1
  4. 2025On the Koopman-Based Generalization Bounds for Multi-Task Deep Learningcited by 1
  5. 2026Operator-Theoretic Generalization Bounds for Multitask Deep Learningcited by 1
  6. 2024Glocal Hypergradient Estimation with Koopman Operatorcited by 0
  7. 2024Deep Koopman-layered Model with Universal Property Based on Toeplitz Matricescited by 0
  8. 2026Unified generalization analysis for physics informed neural networkscited by 0
  9. 2026Sample-Weighted End-to-End Trace-Norm Geometry for Multitask Learningcited by 0