# Neural network layers as parametric spans

arXiv: https://arxiv.org/abs/2208.00809 · HTML: https://mlsanity.com/paper/2208.00809/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 17 references that are in the map.
- First posted on arXiv 1 Aug 2022.

## Builds on (most cited first, 8 of 8)

- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Automatic differentiation in machine learning: a survey](https://mlsanity.com/paper/1502.05767/) (2015)
- [Geometric deep learning on graphs and manifolds using mixture model CNNs](https://mlsanity.com/paper/1611.08402/) (2016)
- [Optimizing Millions of Hyperparameters by Implicit Differentiation](https://mlsanity.com/paper/1911.02590/) (2019)
- [Diffusion-Convolutional Neural Networks](https://mlsanity.com/paper/1511.02136/) (2015)
- [Learning shape correspondence with anisotropic convolutional neural networks](https://mlsanity.com/paper/1605.06437/) (2016)
- [Parametric machines: a fresh approach to architecture search](https://mlsanity.com/paper/2007.02777/) (2020)
- [Machines of finite depth: towards a formalization of neural networks](https://mlsanity.com/paper/2204.12786/) (2022)

## Built on by (most cited first, 2 of 2)

- [Persistence-based operators in machine learning](https://mlsanity.com/paper/2212.13985/) (2022)
- [Disentangling Hyperedges through the Lens of Category Theory](https://mlsanity.com/paper/2510.16289/) (2025)

## Abstract (arXiv)

Properties such as composability and automatic differentiation made artificial neural networks a pervasive tool in applications. Tackling more challenging problems caused neural networks to progressively become more complex and thus difficult to define from a mathematical perspective. We present a general definition of linear layer arising from a categorical framework based on the notions of integration theory and parametric spans. This definition generalizes and encompasses classical layers (e.g., dense, convolutional), while guaranteeing existence and computability of the layer's derivatives for backpropagation.