# Do We Need an Encoder-Decoder to Model Dynamical Systems on Networks?

arXiv: https://arxiv.org/abs/2305.12185 · HTML: https://mlsanity.com/paper/2305.12185/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 53 references that are in the map.
- First posted on arXiv 20 May 2023.

## Builds on (most cited first, 16 of 16)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Deep Sets](https://mlsanity.com/paper/1703.06114/) (2017)
- [Augmented Neural ODEs](https://mlsanity.com/paper/1904.01681/) (2019)
- [GRAND: Graph Neural Diffusion](https://mlsanity.com/paper/2106.10934/) (2021)
- [Dissecting Neural ODEs](https://mlsanity.com/paper/2002.08071/) (2020)
- [Stiff Neural Ordinary Differential Equations](https://mlsanity.com/paper/2103.15341/) (2021)
- [Approximation Capabilities of Neural ODEs and Invertible Residual Networks](https://mlsanity.com/paper/1907.12998/) (2019)
- [Parameterized Neural Ordinary Differential Equations: Applications to Computational Physics Problems](https://mlsanity.com/paper/2010.14685/) (2020)
- [Neural Dynamics on Complex Networks](https://mlsanity.com/paper/1908.06491/) (2019)
- [Climate Modeling with Neural Diffusion Equations](https://mlsanity.com/paper/2111.06011/) (2021)
- [Evolutionary Preference Learning via Graph Nested GRU ODE for Session-based Recommendation](https://mlsanity.com/paper/2206.12779/) (2022)
- [Unsupervised Deep Haar Scattering on Graphs](https://mlsanity.com/paper/1406.2390/) (2014)
- [ConTIG: Continuous Representation Learning on Temporal Interaction Graphs](https://mlsanity.com/paper/2110.06088/) (2021)

## Built on by (most cited first, 3 of 3)

- [Temporal Graph ODEs for Irregularly-Sampled Time Series](https://mlsanity.com/paper/2404.19508/) (2024)
- [Learning Interpretable Network Dynamics via Universal Neural Symbolic Regression](https://mlsanity.com/paper/2411.06833/) (2024)
- [Learning Continuous Network Emerging Dynamics from Scarce Observations via Data-Adaptive Stochastic Processes](https://mlsanity.com/paper/2310.16466/) (2023)

## Abstract (arXiv)

As deep learning gains popularity in modelling dynamical systems, we expose an underappreciated misunderstanding relevant to modelling dynamics on networks. Strongly influenced by graph neural networks, latent vertex embeddings are naturally adopted in many neural dynamical network models. However, we show that embeddings tend to induce a model that fits observations well but simultaneously has incorrect dynamical behaviours. Recognising that previous studies narrowly focus on short-term predictions during the transient phase of a flow, we propose three tests for correct long-term behaviour, and illustrate how an embedding-based dynamical model fails these tests, and analyse the causes, particularly through the lens of topological conjugacy. In doing so, we show that the difficulties can be avoided by not using embedding. We propose a simple embedding-free alternative based on parametrising two additive vector-field components. Through extensive experiments, we verify that the proposed model can reliably recover a broad class of dynamics on different network topologies from time series data.