# Learning Linear Embeddings for Non-Linear Network Dynamics with Koopman Message Passing

arXiv: https://arxiv.org/abs/2305.09060 · HTML: https://mlsanity.com/paper/2305.09060/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 39 references that are in the map.
- First posted on arXiv 15 May 2023.

## Builds on (most cited first, 23 of 23)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Neural Message Passing for Quantum Chemistry](https://mlsanity.com/paper/1704.01212/) (2017)
- [Masked Label Prediction: Unified Message Passing Model for Semi-Supervised Classification](https://mlsanity.com/paper/2009.03509/) (2020)
- [Deep learning for universal linear embeddings of nonlinear dynamics](https://mlsanity.com/paper/1712.09707/) (2017)
- [Modern Koopman Theory for Dynamical Systems](https://mlsanity.com/paper/2102.12086/) (2021)
- [Gradient Descent Happens in a Tiny Subspace](https://mlsanity.com/paper/1812.04754/) (2018)
- [Linearly-Recurrent Autoencoder Networks for Learning Dynamics](https://mlsanity.com/paper/1712.01378/) (2017)
- [Sparse identification of nonlinear dynamics for model predictive control in the low-data limit](https://mlsanity.com/paper/1711.05501/) (2017)
- [Learning Deep Neural Network Representations for Koopman Operators of Nonlinear Dynamical Systems](https://mlsanity.com/paper/1708.06850/) (2017)
- [Forecasting Sequential Data using Consistent Koopman Autoencoders](https://mlsanity.com/paper/2003.02236/) (2020)
- [Learning Compositional Koopman Operators for Model-Based Control](https://mlsanity.com/paper/1910.08264/) (2019)
- [Optimizing Neural Networks via Koopman Operator Theory](https://mlsanity.com/paper/2006.02361/) (2020)
- [Deep Learning of Koopman Representation for Control](https://mlsanity.com/paper/2010.07546/) (2020)
- [Deep Koopman Operator with Control for Nonlinear Systems](https://mlsanity.com/paper/2202.08004/) (2022)
- [Extended Dynamic Mode Decomposition with Learned Koopman Eigenfunctions for Prediction and Control](https://mlsanity.com/paper/1911.08751/) (2019)
- [Dynamic mode decomposition in vector-valued reproducing kernel Hilbert spaces for extracting dynamical structure among observables](https://mlsanity.com/paper/1808.10551/) (2018)
- [Applications of Koopman Mode Analysis to Neural Networks](https://mlsanity.com/paper/2006.11765/) (2020)
- [Accelerating Training in Artificial Neural Networks with Dynamic Mode Decomposition](https://mlsanity.com/paper/2006.14371/) (2020)
- [Dynamical Systems and Neural Networks](https://mlsanity.com/paper/2004.11826/) (2020)
- [Learning to Optimize with Dynamic Mode Decomposition](https://mlsanity.com/paper/2211.16268/) (2022)
- [The Underlying Correlated Dynamics in Neural Training](https://mlsanity.com/paper/2212.09040/) (2022)

## Built on by (most cited first, 1 of 1)

- [Enhancing Neural Training via a Correlated Dynamics Model](https://mlsanity.com/paper/2312.13247/) (2023)

## Abstract (arXiv)

Recently, Koopman operator theory has become a powerful tool for developing linear representations of non-linear dynamical systems. However, existing data-driven applications of Koopman operator theory, including both traditional and deep learning approaches, perform poorly on non-linear network dynamics problems as they do not address the underlying geometric structure. In this paper we present a novel approach based on Koopman operator theory and message passing networks that finds a linear representation for the dynamical system which is globally valid at any time step. The linearisations found by our method produce predictions on a suite of network dynamics problems that are several orders of magnitude better than current state-of-the-art techniques. We also apply our approach to the highly non-linear training dynamics of neural network architectures, and obtain linear representations which can generate network parameters with comparable performance to networks trained by classical optimisers.