# Reverse Derivative Ascent: A Categorical Approach to Learning Boolean Circuits

arXiv: https://arxiv.org/abs/2101.10488 · HTML: https://mlsanity.com/paper/2101.10488/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 22 references that are in the map.
- First posted on arXiv 26 Jan 2021.

## Builds on (most cited first, 10 of 10)

- [An overview of gradient descent optimization algorithms](https://mlsanity.com/paper/1609.04747/) (2016)
- [BinaryConnect: Training Deep Neural Networks with binary weights during propagations](https://mlsanity.com/paper/1511.00363/) (2015)
- [Backprop as Functor: A compositional perspective on supervised learning](https://mlsanity.com/paper/1711.10455/) (2017)
- [LUTNet: Rethinking Inference in FPGA Soft Logic](https://mlsanity.com/paper/1904.00938/) (2019)
- [Reverse derivative categories](https://mlsanity.com/paper/1910.07065/) (2019)
- [Learning Functors using Gradient Descent](https://mlsanity.com/paper/2009.06837/) (2020)
- [Differentiable Causal Computations via Delayed Trace](https://mlsanity.com/paper/1903.01093/) (2019)
- [Rewriting modulo symmetric monoidal structure](https://mlsanity.com/paper/1602.06771/) (2016)
- [Rewriting in Free Hypergraph Categories](https://mlsanity.com/paper/1712.09495/) (2017)
- [The differential calculus of causal functions](https://mlsanity.com/paper/1904.10611/) (2019)

## Built on by (most cited first, 10 of 10)

- [Categorical Foundations of Gradient-Based Learning](https://mlsanity.com/paper/2103.01931/) (2021)
- [Category Theory in Machine Learning](https://mlsanity.com/paper/2106.07032/) (2021)
- [Categorical Foundations of Explainable AI: A Unifying Theory](https://mlsanity.com/paper/2304.14094/) (2023)
- [Categories of Differentiable Polynomial Circuits for Machine Learning](https://mlsanity.com/paper/2203.06430/) (2022)
- [Pooling Image Datasets With Multiple Covariate Shift and Imbalance](https://mlsanity.com/paper/2403.02598/) (2024)
- [Deep Learning with Parametric Lenses](https://mlsanity.com/paper/2404.00408/) (2024)
- [Towards a Categorical Foundation of Deep Learning: A Survey](https://mlsanity.com/paper/2410.05353/) (2024)
- [Generalized Optimization: A First Step Towards Category Theoretic Learning Theory](https://mlsanity.com/paper/2109.10262/) (2021)
- [Reverse Fa`a di Bruno's Formula for Cartesian Reverse Differential Categories](https://mlsanity.com/paper/2509.20931/) (2025)
- [Convergence for Discrete Parameter Update Schemes](https://mlsanity.com/paper/2512.04051/) (2025)

## Abstract (arXiv)

We introduce Reverse Derivative Ascent: a categorical analogue of gradient based methods for machine learning. Our algorithm is defined at the level of so-called reverse differential categories. It can be used to learn the parameters of models which are expressed as morphisms of such categories. Our motivating example is boolean circuits: we show how our algorithm can be applied to such circuits by using the theory of reverse differential categories. Note our methodology allows us to learn the parameters of boolean circuits directly, in contrast to existing binarised neural network approaches. Moreover, we demonstrate its empirical value by giving experimental results on benchmark machine learning datasets.