# Gradient flows on graphons: existence, convergence, continuity equations

arXiv: https://arxiv.org/abs/2111.09459 · HTML: https://mlsanity.com/paper/2111.09459/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 66 references that are in the map.
- First posted on arXiv 18 Nov 2021.

## Builds on (most cited first, 8 of 8)

- [A Mean Field View of the Landscape of Two-Layers Neural Networks](https://mlsanity.com/paper/1804.06561/) (2018)
- [On the Global Convergence of Gradient Descent for Over-parameterized Models using Optimal Transport](https://mlsanity.com/paper/1805.09545/) (2018)
- [Mean-field theory of two-layers neural networks: dimension-free bounds and kernel limit](https://mlsanity.com/paper/1902.06015/) (2019)
- [Mean Field Analysis of Neural Networks: A Central Limit Theorem](https://mlsanity.com/paper/1808.09372/) (2018)
- [A Rigorous Framework for the Mean Field Limit of Multilayer Neural Networks](https://mlsanity.com/paper/2001.11443/) (2020)
- [Gradient Descent on Infinitely Wide Neural Networks: Global Convergence and Generalization](https://mlsanity.com/paper/2110.08084/) (2021)
- [Exponential random graphs behave like mixtures of stochastic block models](https://mlsanity.com/paper/1707.01227/) (2017)
- [Ordered Graph Limits and Their Applications](https://mlsanity.com/paper/1811.02023/) (2018)

## Built on by (most cited first, 3 of 3)

- [Infinite-width limit of deep linear neural networks](https://mlsanity.com/paper/2211.16980/) (2022)
- [Stochastic optimization on matrices and a graphon McKean-Vlasov limit](https://mlsanity.com/paper/2210.00422/) (2022)
- [Path convergence of Markov chains on large graphs](https://mlsanity.com/paper/2308.09214/) (2023)

## Abstract (arXiv)

Wasserstein gradient flows on probability measures have found a host of applications in various optimization problems. They typically arise as the continuum limit of exchangeable particle systems evolving by some mean-field interaction involving a gradient-type potential. However, in many problems, such as in multi-layer neural networks, the so-called particles are edge weights on large graphs whose nodes are exchangeable. Such large graphs are known to converge to continuum limits called graphons as their size grow to infinity. We show that the Euclidean gradient flow of a suitable function of the edge-weights converges to a novel continuum limit given by a curve on the space of graphons that can be appropriately described as a gradient flow or, more technically, a curve of maximal slope. Several natural functions on graphons, such as homomorphism functions and the scalar entropy, are covered by our set-up, and the examples have been worked out in detail.