mlsanityData to 14 Oct 2026

Reverse derivative categories

arXiv 1910.07065 ↗First posted 15 Oct 2019cs.LO math.CTcited by 15

Not on the map. Why?

Abstract (arXiv)

The reverse derivative is a fundamental operation in machine learning and automatic differentiation. This paper gives a direct axiomatization of a category with a reverse derivative operation, in a similar style to that given by Cartesian differential categories for a forward derivative. Intriguingly, a category with a reverse derivative also has a forward derivative, but the converse is not true. In fact, we show explicitly what a forward derivative is missing: a reverse derivative is equivalent to a forward derivative with a dagger structure on its subcategory of linear maps. Furthermore, we show that these linear maps form an additively enriched category with dagger biproducts.

Key facts. Cited by 15 papers in the map (data through 14 Oct 2026). First posted on arXiv 15 Oct 2019.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 6; 2022: 3; 2023: 1; 2024: 4; 2025: 1.

15 papers.

Sort:

  1. 2021Categorical Foundations of Gradient-Based Learningcited by 44
  2. 2021Category Theory in Machine Learningcited by 36
  3. 2021Reverse Derivative Ascent: A Categorical Approach to Learning Boolean Circuitscited by 10
  4. 2024Neural Circuit Diagrams: Robust Diagrams for the Communication, Implementation, and Analysis of Deep Learning Architecturescited by 8
  5. 2021Diagrammatic Differentiation for Quantum Machine Learningcited by 5
  6. 2022Categories of Differentiable Polynomial Circuits for Machine Learningcited by 4
  7. 2023ωPAP Spaces: Reasoning Denotationally About Higher-Order, Recursive Probabilistic and Differentiable Programscited by 3
  8. 2024Deep Learning with Parametric Lensescited by 3
  9. 2021Functorial String Diagrams for Reverse-Mode Automatic Differentiationcited by 2
  10. 2024Towards a Categorical Foundation of Deep Learning: A Surveycited by 2
  11. 2021Generalized Optimization: A First Step Towards Category Theoretic Learning Theorycited by 1
  12. 2022Combinatory Adjoints and Differentiationcited by 0
3 more
  1. 2022Graph Convolutional Neural Networks as Parametric CoKleisli morphismscited by 0
  2. 2024Generalized Gradient Descent is a Hypergraph Functorcited by 0
  3. 2025Reverse Fa`a di Bruno's Formula for Cartesian Reverse Differential Categoriescited by 0