# A Simple Differentiable Programming Language

arXiv: https://arxiv.org/abs/1911.04523 · HTML: https://mlsanity.com/paper/1911.04523/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Nov 2019.

## Built on by (most cited first, 11 of 11)

- [Opening the Blackbox: Accelerating Neural Differential Equations by Regularizing Internal Solver Heuristics](https://mlsanity.com/paper/2105.03918/) (2021)
- [Differentiable Physics: A Position Piece](https://mlsanity.com/paper/2109.07573/) (2021)
- [Smoothing Methods for Automatic Differentiation Across Conditional Branches](https://mlsanity.com/paper/2310.03585/) (2023)
- [On the Correctness of Automatic Differentiation for Neural Networks with Machine-Representable Parameters](https://mlsanity.com/paper/2301.13370/) (2023)
- [Auto-Differentiation of Relational Computations for Very Large Scale Machine Learning](https://mlsanity.com/paper/2306.00088/) (2023)
- [Learning to Optimize by Differentiable Programming](https://mlsanity.com/paper/2601.16510/) (2026)
- [On Reinforcement Learning, Effect Handlers, and the State Monad](https://mlsanity.com/paper/2203.15426/) (2022)
- [Differentiable Quantum Programming with Unbounded Loops](https://mlsanity.com/paper/2211.04507/) (2022)
- [Efficient and Sound Differentiable Programming in a Functional Array-Processing Language](https://mlsanity.com/paper/2212.10307/) (2022)
- [Compiling to recurrent neurons](https://mlsanity.com/paper/2511.14953/) (2025)
- [Differentiate the Evaluator, Not the Program: An Efficient Runtime Representation for Neuro-Symbolic Learning](https://mlsanity.com/paper/2607.03574/) (2026)

## Abstract (arXiv)

Automatic differentiation plays a prominent role in scientific computing and in modern machine learning, often in the context of powerful programming systems. The relation of the various embodiments of automatic differentiation to the mathematical notion of derivative is not always entirely clear---discrepancies can arise, sometimes inadvertently. In order to study automatic differentiation in such programming contexts, we define a small but expressive programming language that includes a construct for reverse-mode differentiation. We give operational and denotational semantics for this language. The operational semantics employs popular implementation techniques, while the denotational semantics employs notions of differentiation familiar from real analysis. We establish that these semantics coincide.