mlsanityData to 14 Oct 2026

Backprop as Functor: A compositional perspective on supervised learning

arXiv 1711.10455 ↗First posted 28 Nov 2017math.CT cs.AI cs.LGcited by 44

Not on the map. Why?

Abstract (arXiv)

A supervised learning algorithm searches over a set of functions A → B parametrised by a space P to find the best approximation to some ideal function fcolon A → B. It does this by taking examples (a,f(a)) ∈ A× B, and updating the parameter according to some rule. We define a category where these update rules may be composed, and show that gradient descent---with respect to a fixed step size and an error function satisfying a certain property---defines a monoidal functor from a category of parametrised functions to this category of update rules. This provides a structural perspective on backpropagation, as well as a broad generalisation of neural networks.

Key facts. Cited by 44 papers in the map (data through 14 Oct 2026). First posted on arXiv 28 Nov 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 8; 2022: 1; 2023: 4; 2024: 9; 2025: 12; 2026: 10.

44 papers.

Sort:

By area:

  1. 2024Position: Topological Deep Learning is the New Frontier for Relational Learningcited by 66
  2. 2021Categorical Foundations of Gradient-Based Learningcited by 44
  3. 2021Category Theory in Machine Learningcited by 36
  4. 2024Position: Categorical Deep Learning is an Algebraic Theory of All Architecturescited by 32
  5. 2023Going Beyond Neural Network Feature Similarity: The Network Feature Complexity and Its Interpretation Using Category Theorycited by 12
  6. 2021Reverse Derivative Ascent: A Categorical Approach to Learning Boolean Circuitscited by 10
  7. 2024Neural Circuit Diagrams: Robust Diagrams for the Communication, Implementation, and Analysis of Deep Learning Architecturescited by 8
  8. 2025Large Causal Models from Large Language Modelscited by 8
  9. 2024FlashAttention on a Napkin: A Diagrammatic Approach to Deep Learning IO-Awarenesscited by 7
  10. 2026Identifiable Equivariant Networks are Layerwise Equivariantcited by 7
  11. 2021Learners' Languagescited by 6
  12. 2023On Meta-Promptingcited by 6
32 more
  1. 2021Topos and Stacks of Deep Neural Networkscited by 5
  2. 2024Pooling Image Datasets With Multiple Covariate Shift and Imbalancecited by 4
  3. 2026Self-Revising Discovery Systems for Science: A Categorical Framework for Agentic Artificial Intelligencecited by 4
  4. 2023Active Inference in String Diagrams: A Categorical Account of Predictive Processing and Free Energycited by 3
  5. 2024Deep Learning with Parametric Lensescited by 3
  6. 2024Token Space: A Category Theory Framework for AI Computationscited by 2
  7. 2025Topos Theory for Generative AI and LLMscited by 2
  8. 2025Universal Reinforcement Learning in Coalgebras: Asynchronous Stochastic Computation via Conductioncited by 2
  9. 2021Generalized Optimization: A First Step Towards Category Theoretic Learning Theorycited by 1
  10. 2021Deep neural networks as nested dynamical systemscited by 1
  11. 2021A category theory framework for Bayesian learningcited by 1
  12. 2024The Topos of Transformer Networkscited by 1
  13. 2025Accelerating Machine Learning Systems via Category Theory: Applications to Spherical Attention for Gene Regulatory Networkscited by 1
  14. 2025A Rose by Any Other Name Would Smell as Sweet: Categorical Homotopy Theory for Large Language Modelscited by 1
  15. 2025Jet Functors and Weil Algebras in Automatic Differentiation: A Geometric Analysiscited by 1
  16. 2026Learning in Infinitesimal Non-Compositional Sketchescited by 1
  17. 2022A Probabilistic Generative Model of Free Categoriescited by 0
  18. 2023Algebraic Dynamical Systems in Machine Learningcited by 0
  19. 2024Metric Learning for Clifford Group Equivariant Neural Networkscited by 0
  20. 2025The Gauss-Markov Adjunction Provides Categorical Semantics of Residuals in Supervised Learningcited by 0
  21. 2025Higher Gauge Flow Modelscited by 0
  22. 2025Topos Causal Modelscited by 0
  23. 2025Transparent Semantic Spaces: A Categorical Approach to Explainable Word Embeddingscited by 0
  24. 2025Reverse Fa`a di Bruno's Formula for Cartesian Reverse Differential Categoriescited by 0
  25. 2025Disentangling Hyperedges through the Lens of Category Theorycited by 0
  26. 2026CSQL: Mapping Documents into Causal Databasescited by 0
  27. 2026Coalgebras for categorical deep learning: Representability and universal approximationcited by 0
  28. 2026Functorial Neural Architectures from Higher Inductive Typescited by 0
  29. 2026From Intent to Evidence: A Categorical Approach for Structural Evaluation of Deep Research Agentscited by 0
  30. 2026Weaves, Wires, and Morphisms: Formalizing and Implementing the Algebra of Deep Learningcited by 0
  31. 2026NeSyCat Torch: A Differentiable Tensor Implementation of Categorical Semantics for Neurosymbolic Learningcited by 0
  32. 2026When the Canonical Completion Is Wrong: Formalizing and Measuring the Jump in Large Language Modelscited by 0