# From Continuous Dynamics to Graph Neural Networks: Neural Diffusion and Beyond

arXiv: https://arxiv.org/abs/2310.10121 · HTML: https://mlsanity.com/paper/2310.10121/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- Builds on 83 of its 127 references that are in the map.
- First posted on arXiv 16 Oct 2023.

## Builds on (most cited first, 30 of 83)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [How Powerful are Graph Neural Networks?](https://mlsanity.com/paper/1810.00826/) (2018)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Neural Message Passing for Quantum Chemistry](https://mlsanity.com/paper/1704.01212/) (2017)
- [Open Graph Benchmark: Datasets for Machine Learning on Graphs](https://mlsanity.com/paper/2005.00687/) (2020)
- [Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering](https://mlsanity.com/paper/1606.09375/) (2016)
- [DeepWalk: Online Learning of Social Representations](https://mlsanity.com/paper/1403.6652/) (2014)
- [Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges](https://mlsanity.com/paper/2104.13478/) (2021)
- [Geometric deep learning: going beyond Euclidean data](https://mlsanity.com/paper/1611.08097/) (2016)
- [Pitfalls of Graph Neural Network Evaluation](https://mlsanity.com/paper/1811.05868/) (2018)
- [Weisfeiler and Leman Go Neural: Higher-order Graph Neural Networks](https://mlsanity.com/paper/1810.02244/) (2018)
- [Geom-GCN: Geometric Graph Convolutional Networks](https://mlsanity.com/paper/2002.05287/) (2020)
- [Gated Graph Sequence Neural Networks](https://mlsanity.com/paper/1511.05493/) (2015)
- [Heterogeneous Graph Attention Network](https://mlsanity.com/paper/1903.07293/) (2019)
- [Neural Operator: Graph Kernel Network for Partial Differential Equations](https://mlsanity.com/paper/2003.03485/) (2020)
- [Graph Neural Networks for Social Recommendation](https://mlsanity.com/paper/1902.07243/) (2019)
- [Image-based Recommendations on Styles and Substitutes](https://mlsanity.com/paper/1506.04757/) (2015)
- [Equivariant Diffusion for Molecule Generation in 3D](https://mlsanity.com/paper/2203.17003/) (2022)
- [A Survey on Knowledge Graphs: Representation, Acquisition and Applications](https://mlsanity.com/paper/2002.00388/) (2020)
- [On the Bottleneck of Graph Neural Networks and its Practical Implications](https://mlsanity.com/paper/2006.05205/) (2020)
- [Multi-scale Attributed Node Embedding](https://mlsanity.com/paper/1909.13021/) (2019)
- [Understanding over-squashing and bottlenecks on graphs via curvature](https://mlsanity.com/paper/2111.14522/) (2021)
- [Diffusion Improves Graph Learning](https://mlsanity.com/paper/1911.05485/) (2019)
- [Hyperbolic Graph Convolutional Neural Networks](https://mlsanity.com/paper/1910.12933/) (2019)
- [Multipole Graph Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2006.09535/) (2020)
- [Stable Architectures for Deep Neural Networks](https://mlsanity.com/paper/1705.03341/) (2017)

## Built on by (most cited first, 26 of 26)

- [On Oversquashing in Graph Neural Networks Through the Lens of Dynamical Systems](https://mlsanity.com/paper/2405.01009/) (2024)
- [Exposition on over-squashing problem on GNNs: Current Methods, Benchmarks and Challenges](https://mlsanity.com/paper/2311.07073/) (2023)
- [Unleashing the Potential of Fractional Calculus in Graph Neural Networks with FROND](https://mlsanity.com/paper/2404.17099/) (2024)
- [Graph ODEs and Beyond: A Comprehensive Survey on Integrating Differential Equations with Graph Neural Networks](https://mlsanity.com/paper/2503.23167/) (2025)
- [Coupling Graph Neural Networks with Fractional Order Continuous Dynamics: A Robustness Study](https://mlsanity.com/paper/2401.04331/) (2024)
- [Continuous Product Graph Neural Networks](https://mlsanity.com/paper/2405.18877/) (2024)
- [Distributed-Order Fractional Graph Operating Network](https://mlsanity.com/paper/2411.05274/) (2024)
- [Design Your Own Universe: A Physics-Informed Agnostic Method for Enhancing Graph Neural Networks](https://mlsanity.com/paper/2401.14580/) (2024)
- [Improving the Effective Receptive Field of Message-Passing Neural Networks](https://mlsanity.com/paper/2505.23185/) (2025)
- [When Graph Neural Networks Meet Dynamic Mode Decomposition](https://mlsanity.com/paper/2410.05593/) (2024)
- [Implicit Hypergraph Neural Networks: A Stable Framework for Higher-Order Relational Learning with Provable Guarantees](https://mlsanity.com/paper/2508.09427/) (2025)
- [Unleash Graph Neural Networks from Heavy Tuning](https://mlsanity.com/paper/2405.12521/) (2024)
- [Interpreting Temporal Graph Neural Networks with Koopman Theory](https://mlsanity.com/paper/2410.13469/) (2024)
- [Resolving Oversmoothing with Opinion Dissensus](https://mlsanity.com/paper/2501.19089/) (2025)
- [Statistical physics analysis of graph neural networks: Approaching optimality in the contextual stochastic block model](https://mlsanity.com/paper/2503.01361/) (2025)
- [Continuous Simplicial Neural Networks](https://mlsanity.com/paper/2503.12919/) (2025)
- [Integrating GNN and Neural ODEs for Estimating Non-Reciprocal Two-Body Interactions in Mixed-Species Collective Motion](https://mlsanity.com/paper/2405.16503/) (2024)
- [Sigma Flows for Image and Data Labeling and Learning Structured Prediction](https://mlsanity.com/paper/2408.15946/) (2024)
- [Diffusing to the Top: Boost Graph Neural Networks with Minimal Hyperparameter Tuning](https://mlsanity.com/paper/2410.05697/) (2024)
- [SFi-Former: Sparse Flow Induced Attention for Graph Transformer](https://mlsanity.com/paper/2504.20666/) (2025)
- [TANGO: Graph Neural Dynamics via Learned Energy and Tangential Flows](https://mlsanity.com/paper/2508.05070/) (2025)
- [Wavy Transformer](https://mlsanity.com/paper/2508.12787/) (2025)
- [Graph Neural Regularizers for PDE Inverse Problems](https://mlsanity.com/paper/2510.21012/) (2025)
- [A Geometric Perspective on the Difficulties of Learning GNN-based SAT Solvers](https://mlsanity.com/paper/2508.21513/) (2025)
- [Second-Order Tensorial Partial Differential Equations on Graphs](https://mlsanity.com/paper/2509.02015/) (2025)
- [Remedying Coarsening-Based GNN Training under Heterophily via Adaptive Complementary Enhancement](https://mlsanity.com/paper/2607.21885/) (2026)

## Abstract (arXiv)

Graph neural networks (GNNs) have demonstrated significant promise in modelling relational data and have been widely applied in various fields of interest. The key mechanism behind GNNs is the so-called message passing where information is being iteratively aggregated to central nodes from their neighbourhood. Such a scheme has been found to be intrinsically linked to a physical process known as heat diffusion, where the propagation of GNNs naturally corresponds to the evolution of heat density. Analogizing the process of message passing to the heat dynamics allows to fundamentally understand the power and pitfalls of GNNs and consequently informs better model design. Recently, there emerges a plethora of works that proposes GNNs inspired from the continuous dynamics formulation, in an attempt to mitigate the known limitations of GNNs, such as oversmoothing and oversquashing. In this survey, we provide the first systematic and comprehensive review of studies that leverage the continuous perspective of GNNs. To this end, we introduce foundational ingredients for adapting continuous dynamics to GNNs, along with a general framework for the design of graph neural dynamics. We then review and categorize existing works based on their driven mechanisms and underlying dynamics. We also summarize how the limitations of classic GNNs can be addressed under the continuous framework. We conclude by identifying multiple open research directions.