# Differentiable Causal Computations via Delayed Trace

arXiv: https://arxiv.org/abs/1903.01093 · HTML: https://mlsanity.com/paper/1903.01093/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Mar 2019.

## Built on by (most cited first, 5 of 5)

- [Categorical Foundations of Gradient-Based Learning](https://mlsanity.com/paper/2103.01931/) (2021)
- [Category Theory in Machine Learning](https://mlsanity.com/paper/2106.07032/) (2021)
- [Reverse Derivative Ascent: A Categorical Approach to Learning Boolean Circuits](https://mlsanity.com/paper/2101.10488/) (2021)
- [Categorical Foundations of Explainable AI: A Unifying Theory](https://mlsanity.com/paper/2304.14094/) (2023)
- [Deep Learning with Parametric Lenses](https://mlsanity.com/paper/2404.00408/) (2024)

## Abstract (arXiv)

We investigate causal computations taking sequences of inputs to sequences of outputs where the nth output depends on the first n inputs only. We model these in category theory via a construction taking a Cartesian category C to another category St(C) with a novel trace-like operation called "delayed trace", which misses yanking and dinaturality axioms of the usual trace. The delayed trace operation provides a feedback mechanism in St(C) with an implicit guardedness guarantee. When C is equipped with a Cartesian differential operator, we construct a differential operator for St(C) using an abstract version of backpropagation through time, a technique from machine learning based on unrolling of functions. This obtains a swath of properties for backpropagation through time, including a chain rule and Schwartz theorem. Our differential operator is also able to compute the derivative of a stateful network without requiring the network to be unrolled.