mlsanityData to 7 Oct 2026

Decomposing reverse-mode automatic differentiation

arXiv 2105.09469 ↗First posted 20 May 2021cs.PL cs.LGcited by 4

Not on the map. Why?

Abstract (arXiv)

We decompose reverse-mode automatic differentiation into (forward-mode) linearization followed by transposition. Doing so isolates the essential difference between forward- and reverse-mode AD, and simplifies their joint implementation. In particular, once forward-mode AD rules are defined for every primitive operation in a source language, only linear primitives require an additional transposition rule in order to arrive at a complete reverse-mode AD implementation. This is how reverse-mode AD is written in JAX and Dex.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 5 of its 12 references that are in the map. First posted on arXiv 20 May 2021.

Builds on

5 of 12 references.

References · 5

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2015Automatic differentiation in machine learning: a surveycited by 846
  3. 2018The simple essence of automatic differentiationcited by 22
  4. 2021Getting to the Point. Index Sets and Parallelism-Preserving Autodiff for Pointful Array Programmingcited by 10
  5. 2016DiffSharp: An AD Library for .NET Languagescited by 3

Built on by

Cited by year

Citing papers per year: 2021: 2; 2022: 2.

4 papers.

  1. 2021Efficient and Modular Implicit Differentiationcited by 199
  2. 2021Equinox: neural networks in JAX via callable PyTrees and filtered transformationscited by 143
  3. 2022Fast Finite Width Neural Tangent Kernelcited by 62
  4. 2022Neural Bregman Divergences for Distance Learningcited by 3