# Reverse derivative categories

arXiv: https://arxiv.org/abs/1910.07065 · HTML: https://mlsanity.com/paper/1910.07065/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Oct 2019.

## Built on by (most cited first, 15 of 15)

- [Categorical Foundations of Gradient-Based Learning](https://mlsanity.com/paper/2103.01931/) (2021)
- [Category Theory in Machine Learning](https://mlsanity.com/paper/2106.07032/) (2021)
- [Reverse Derivative Ascent: A Categorical Approach to Learning Boolean Circuits](https://mlsanity.com/paper/2101.10488/) (2021)
- [Neural Circuit Diagrams: Robust Diagrams for the Communication, Implementation, and Analysis of Deep Learning Architectures](https://mlsanity.com/paper/2402.05424/) (2024)
- [Diagrammatic Differentiation for Quantum Machine Learning](https://mlsanity.com/paper/2103.07960/) (2021)
- [Categories of Differentiable Polynomial Circuits for Machine Learning](https://mlsanity.com/paper/2203.06430/) (2022)
- [ωPAP Spaces: Reasoning Denotationally About Higher-Order, Recursive Probabilistic and Differentiable Programs](https://mlsanity.com/paper/2302.10636/) (2023)
- [Deep Learning with Parametric Lenses](https://mlsanity.com/paper/2404.00408/) (2024)
- [Functorial String Diagrams for Reverse-Mode Automatic Differentiation](https://mlsanity.com/paper/2107.13433/) (2021)
- [Towards a Categorical Foundation of Deep Learning: A Survey](https://mlsanity.com/paper/2410.05353/) (2024)
- [Generalized Optimization: A First Step Towards Category Theoretic Learning Theory](https://mlsanity.com/paper/2109.10262/) (2021)
- [Combinatory Adjoints and Differentiation](https://mlsanity.com/paper/2207.00847/) (2022)
- [Graph Convolutional Neural Networks as Parametric CoKleisli morphisms](https://mlsanity.com/paper/2212.00542/) (2022)
- [Generalized Gradient Descent is a Hypergraph Functor](https://mlsanity.com/paper/2403.19845/) (2024)
- [Reverse Fa`a di Bruno's Formula for Cartesian Reverse Differential Categories](https://mlsanity.com/paper/2509.20931/) (2025)

## Abstract (arXiv)

The reverse derivative is a fundamental operation in machine learning and automatic differentiation. This paper gives a direct axiomatization of a category with a reverse derivative operation, in a similar style to that given by Cartesian differential categories for a forward derivative. Intriguingly, a category with a reverse derivative also has a forward derivative, but the converse is not true. In fact, we show explicitly what a forward derivative is missing: a reverse derivative is equivalent to a forward derivative with a dagger structure on its subcategory of linear maps. Furthermore, we show that these linear maps form an additively enriched category with dagger biproducts.