# Software-based Automatic Differentiation is Flawed

arXiv: https://arxiv.org/abs/2305.03863 · HTML: https://mlsanity.com/paper/2305.03863/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 15 references that are in the map.
- First posted on arXiv 5 May 2023.

## Builds on (most cited first, 5 of 5)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems](https://mlsanity.com/paper/1603.04467/) (2016)
- [Caffe: Convolutional Architecture for Fast Feature Embedding](https://mlsanity.com/paper/1408.5093/) (2014)
- [Theano: A Python framework for fast computation of mathematical expressions](https://mlsanity.com/paper/1605.02688/) (2016)
- [Analytically Integratable Zero-restlength Springs for Capturing Dynamic Modes unrepresented by Quasistatic Neural Networks](https://mlsanity.com/paper/2201.10122/) (2022)

## Built on by (most cited first, 3 of 3)

- [Neural-Integrated Meshfree (NIM) Method: A differentiable programming-based hybrid solver for computational mechanics](https://mlsanity.com/paper/2311.12915/) (2023)
- [Addressing Discontinuous Root-Finding for Subsequent Differentiability in Machine Learning, Inverse Problems, and Control](https://mlsanity.com/paper/2306.12413/) (2023)
- [Zero Coordinate Shift: Whetted Automatic Differentiation for Physics-informed Operator Learning](https://mlsanity.com/paper/2311.00860/) (2023)

## Abstract (arXiv)

Various software efforts embrace the idea that object oriented programming enables a convenient implementation of the chain rule, facilitating so-called automatic differentiation via backpropagation. Such frameworks have no mechanism for simplifying the expressions (obtained via the chain rule) before evaluating them. As we illustrate below, the resulting errors tend to be unbounded.