# Functorial String Diagrams for Reverse-Mode Automatic Differentiation

arXiv: https://arxiv.org/abs/2107.13433 · HTML: https://mlsanity.com/paper/2107.13433/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 64 references that are in the map.
- First posted on arXiv 28 Jul 2021.

## Builds on (most cited first, 16 of 16)

- [Automatic differentiation in machine learning: a survey](https://mlsanity.com/paper/1502.05767/) (2015)
- [The simple essence of automatic differentiation](https://mlsanity.com/paper/1804.00746/) (2018)
- [Reverse derivative categories](https://mlsanity.com/paper/1910.07065/) (2019)
- [Demystifying Differentiable Programming: Shift/Reset the Penultimate Backpropagator](https://mlsanity.com/paper/1803.10228/) (2018)
- [Correctness of Automatic Differentiation via Diffeologies and Categorical Gluing](https://mlsanity.com/paper/2001.02209/) (2020)
- [Rewriting modulo symmetric monoidal structure](https://mlsanity.com/paper/1602.06771/) (2016)
- [Backpropagation in the Simply Typed Lambda-calculus with Linear Negation](https://mlsanity.com/paper/1909.13768/) (2019)
- [DiffSharp: An AD Library for .NET Languages](https://mlsanity.com/paper/1611.03423/) (2016)
- [Diagrammatic Semantics for Digital Circuits](https://mlsanity.com/paper/1703.10247/) (2017)
- [Reverse AD at Higher Types: Pure, Principled and Denotationally Correct](https://mlsanity.com/paper/2007.05283/) (2020)
- [Normalization for planar string diagrams and a quadratic equivalence algorithm](https://mlsanity.com/paper/1804.07832/) (2018)
- [Proof nets and the call-by-value lambda-calculus](https://mlsanity.com/paper/1303.7326/) (2013)
- [Efficient Implementation of a Higher-Order Language with Built-In AD](https://mlsanity.com/paper/1611.03416/) (2016)
- [Globular: an online proof assistant for higher-dimensional rewriting](https://mlsanity.com/paper/1612.01093/) (2016)
- [The Dynamic Geometry of Interaction Machine: A Token-Guided Graph Rewriter](https://mlsanity.com/paper/1803.00427/) (2018)
- [Operational Semantics with Hierarchical Abstract Syntax Graphs](https://mlsanity.com/paper/2102.02363/) (2021)

## Built on by (most cited first, 2 of 2)

- [Position: Categorical Deep Learning is an Algebraic Theory of All Architectures](https://mlsanity.com/paper/2402.15332/) (2024)
- [Graph Lineages and Skeletal Graph Products](https://mlsanity.com/paper/2508.00197/) (2025)

## Abstract (arXiv)

We enhance the calculus of string diagrams for monoidal categories with hierarchical features in order to capture closed monoidal (and cartesian closed) structure. Using this new syntax we formulate an automatic differentiation algorithm for (applied) simply typed lambda calculus in the style of [Pearlmutter and Siskind 2008] and we prove for the first time its soundness. To give an efficient yet principled implementation of the AD algorithm we define a sound and complete representation of hierarchical string diagrams as a class of hierarchical hypergraphs we call hypernets.