# Backprop as Functor: A compositional perspective on supervised learning

arXiv: https://arxiv.org/abs/1711.10455 · HTML: https://mlsanity.com/paper/1711.10455/

## Key facts

- Cited by 44 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Nov 2017.

## Built on by (most cited first, 30 of 44)

- [Position: Topological Deep Learning is the New Frontier for Relational Learning](https://mlsanity.com/paper/2402.08871/) (2024)
- [Categorical Foundations of Gradient-Based Learning](https://mlsanity.com/paper/2103.01931/) (2021)
- [Category Theory in Machine Learning](https://mlsanity.com/paper/2106.07032/) (2021)
- [Position: Categorical Deep Learning is an Algebraic Theory of All Architectures](https://mlsanity.com/paper/2402.15332/) (2024)
- [Going Beyond Neural Network Feature Similarity: The Network Feature Complexity and Its Interpretation Using Category Theory](https://mlsanity.com/paper/2310.06756/) (2023)
- [Reverse Derivative Ascent: A Categorical Approach to Learning Boolean Circuits](https://mlsanity.com/paper/2101.10488/) (2021)
- [Neural Circuit Diagrams: Robust Diagrams for the Communication, Implementation, and Analysis of Deep Learning Architectures](https://mlsanity.com/paper/2402.05424/) (2024)
- [Large Causal Models from Large Language Models](https://mlsanity.com/paper/2512.07796/) (2025)
- [FlashAttention on a Napkin: A Diagrammatic Approach to Deep Learning IO-Awareness](https://mlsanity.com/paper/2412.03317/) (2024)
- [Identifiable Equivariant Networks are Layerwise Equivariant](https://mlsanity.com/paper/2601.21645/) (2026)
- [Learners' Languages](https://mlsanity.com/paper/2103.01189/) (2021)
- [On Meta-Prompting](https://mlsanity.com/paper/2312.06562/) (2023)
- [Topos and Stacks of Deep Neural Networks](https://mlsanity.com/paper/2106.14587/) (2021)
- [Pooling Image Datasets With Multiple Covariate Shift and Imbalance](https://mlsanity.com/paper/2403.02598/) (2024)
- [Self-Revising Discovery Systems for Science: A Categorical Framework for Agentic Artificial Intelligence](https://mlsanity.com/paper/2606.01444/) (2026)
- [Active Inference in String Diagrams: A Categorical Account of Predictive Processing and Free Energy](https://mlsanity.com/paper/2308.00861/) (2023)
- [Deep Learning with Parametric Lenses](https://mlsanity.com/paper/2404.00408/) (2024)
- [Token Space: A Category Theory Framework for AI Computations](https://mlsanity.com/paper/2404.11624/) (2024)
- [Topos Theory for Generative AI and LLMs](https://mlsanity.com/paper/2508.08293/) (2025)
- [Universal Reinforcement Learning in Coalgebras: Asynchronous Stochastic Computation via Conduction](https://mlsanity.com/paper/2508.15128/) (2025)
- [Generalized Optimization: A First Step Towards Category Theoretic Learning Theory](https://mlsanity.com/paper/2109.10262/) (2021)
- [Deep neural networks as nested dynamical systems](https://mlsanity.com/paper/2111.01297/) (2021)
- [A category theory framework for Bayesian learning](https://mlsanity.com/paper/2111.14293/) (2021)
- [The Topos of Transformer Networks](https://mlsanity.com/paper/2403.18415/) (2024)
- [Accelerating Machine Learning Systems via Category Theory: Applications to Spherical Attention for Gene Regulatory Networks](https://mlsanity.com/paper/2505.09326/) (2025)
- [A Rose by Any Other Name Would Smell as Sweet: Categorical Homotopy Theory for Large Language Models](https://mlsanity.com/paper/2508.10018/) (2025)
- [Jet Functors and Weil Algebras in Automatic Differentiation: A Geometric Analysis](https://mlsanity.com/paper/2510.14342/) (2025)
- [Learning in Infinitesimal Non-Compositional Sketches](https://mlsanity.com/paper/2607.15107/) (2026)
- [A Probabilistic Generative Model of Free Categories](https://mlsanity.com/paper/2205.04545/) (2022)
- [Algebraic Dynamical Systems in Machine Learning](https://mlsanity.com/paper/2311.03118/) (2023)

## Abstract (arXiv)

A supervised learning algorithm searches over a set of functions A → B parametrised by a space P to find the best approximation to some ideal function fcolon A → B. It does this by taking examples (a,f(a)) ∈ A× B, and updating the parameter according to some rule. We define a category where these update rules may be composed, and show that gradient descent---with respect to a fixed step size and an error function satisfying a certain property---defines a monoidal functor from a category of parametrised functions to this category of update rules. This provides a structural perspective on backpropagation, as well as a broad generalisation of neural networks.