# On Non-Linear operators for Geometric Deep Learning

arXiv: https://arxiv.org/abs/2207.03485 · HTML: https://mlsanity.com/paper/2207.03485/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 40 references that are in the map.
- First posted on arXiv 6 Jul 2022.

## Builds on (most cited first, 13 of 13)

- [Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges](https://mlsanity.com/paper/2104.13478/) (2021)
- [Group Equivariant Convolutional Networks](https://mlsanity.com/paper/1602.07576/) (2016)
- [Geometric deep learning: going beyond Euclidean data](https://mlsanity.com/paper/1611.08097/) (2016)
- [Machine Learning for Fluid Mechanics](https://mlsanity.com/paper/1905.11075/) (2019)
- [On the Generalization of Equivariance and Convolution in Neural Networks to the Action of Compact Groups](https://mlsanity.com/paper/1802.03690/) (2018)
- [Gauge Equivariant Convolutional Networks and the Icosahedral CNN](https://mlsanity.com/paper/1902.04615/) (2019)
- [Universal Invariant and Equivariant Graph Neural Networks](https://mlsanity.com/paper/1905.04943/) (2019)
- [Universal approximations of invariant maps by neural networks](https://mlsanity.com/paper/1804.10306/) (2018)
- [Learning with invariances in random features and kernel models](https://mlsanity.com/paper/2102.13219/) (2021)
- [Understanding Deep Convolutional Networks](https://mlsanity.com/paper/1601.04920/) (2016)
- [Coordinate Independent Convolutional Networks -- Isometry and Gauge Equivariant Convolutions on Riemannian Manifolds](https://mlsanity.com/paper/2106.06020/) (2021)
- [Learning the Irreducible Representations of Commutative Lie Groups](https://mlsanity.com/paper/1402.4437/) (2014)
- [Universal Approximation Theorem for Equivariant Maps by Group CNNs](https://mlsanity.com/paper/2012.13882/) (2020)

## Built on by (most cited first, 3 of 3)

- [Test-time Generalization for Physics through Neural Operator Splitting](https://mlsanity.com/paper/2602.00884/) (2026)
- [A tradeoff between universality of equivariant models and learnability of symmetries](https://mlsanity.com/paper/2210.09444/) (2022)
- [Influence of the Geometry of the world model on Curiosity Based Exploration](https://mlsanity.com/paper/2304.00188/) (2023)

## Abstract (arXiv)

This work studies operators mapping vector and scalar fields defined over a manifold M, and which commute with its group of diffeomorphisms Diff(M). We prove that in the case of scalar fields Lpω(M,ℝ), those operators correspond to point-wise non-linearities, recovering and extending known results on ℝd. In the context of Neural Networks defined over M, it indicates that point-wise non-linear operators are the only universal family that commutes with any group of symmetries, and justifies their systematic use in combination with dedicated linear operators commuting with specific symmetries. In the case of vector fields Lpω(M,TM), we show that those operators are solely the scalar multiplication. It indicates that Diff(M) is too rich and that there is no universal class of non-linear operators to motivate the design of Neural Networks over the symmetries of M.