# Don't Unroll Adjoint: Differentiating SSA-Form Programs

arXiv: https://arxiv.org/abs/1810.07951 · HTML: https://mlsanity.com/paper/1810.07951/

## Key facts

- Cited by 45 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Oct 2018.

## Built on by (most cited first, 30 of 45)

- [Interpretable Machine Learning for Science with PySR and SymbolicRegression.jl](https://mlsanity.com/paper/2305.01582/) (2023)
- [Recurrent Equilibrium Networks: Flexible Dynamic Models with Guaranteed Stability and Robustness](https://mlsanity.com/paper/2104.05942/) (2021)
- [Torch.fx: Practical Program Capture and Transformation for Deep Learning in Python](https://mlsanity.com/paper/2112.08429/) (2021)
- [Automatic Differentiation of Programs with Discrete Randomness](https://mlsanity.com/paper/2210.08572/) (2022)
- [Opening the Blackbox: Accelerating Neural Differential Equations by Regularizing Internal Solver Heuristics](https://mlsanity.com/paper/2105.03918/) (2021)
- [Discovering Sparse Interpretable Dynamics from Partial Observations](https://mlsanity.com/paper/2107.10879/) (2021)
- [Physics-enhanced deep surrogates for partial differential equations](https://mlsanity.com/paper/2111.05841/) (2021)
- [Stabilized Neural Differential Equations for Learning Dynamics with Explicit Constraints](https://mlsanity.com/paper/2306.09739/) (2023)
- [Dual Lagrangian Learning for Conic Optimization](https://mlsanity.com/paper/2402.03086/) (2024)
- [PIG: Physics-Informed Gaussians as Adaptive Parametric Mesh Representations](https://mlsanity.com/paper/2412.05994/) (2024)
- [Scalable First-Order Bayesian Optimization via Structured Automatic Differentiation](https://mlsanity.com/paper/2206.08366/) (2022)
- [Combining Pseudo-Point and State Space Approximations for Sum-Separable Gaussian Processes](https://mlsanity.com/paper/2106.10210/) (2021)
- [Closed-Form Diffeomorphic Transformations for Time Series Alignment](https://mlsanity.com/paper/2206.08107/) (2022)
- [Physics-informed machine learning with differentiable programming for heterogeneous underground reservoir pressure management](https://mlsanity.com/paper/2206.10718/) (2022)
- [Learned multiphysics inversion with differentiable programming and machine learning](https://mlsanity.com/paper/2304.05592/) (2023)
- [DiSECt: A Differentiable Simulator for Parameter Inference and Control in Robotic Cutting](https://mlsanity.com/paper/2203.10263/) (2022)
- [Proximal Oracles for Optimization and Sampling](https://mlsanity.com/paper/2404.02239/) (2024)
- [What You See is Not What You Get: Neural Partial Differential Equations and The Illusion of Learning](https://mlsanity.com/paper/2411.15101/) (2024)
- [Branching Flows: Discrete, Continuous, and Manifold Flow Matching with Splits and Deletions](https://mlsanity.com/paper/2511.09465/) (2025)
- [Provably Scalable Black-Box Variational Inference with Structured Variational Families](https://mlsanity.com/paper/2401.10989/) (2024)
- [Sparser, Better, Faster, Stronger: Sparsity Detection for Efficient Automatic Differentiation](https://mlsanity.com/paper/2501.17737/) (2025)
- [Spline parameterization of neural network controls for deep learning](https://mlsanity.com/paper/2103.00301/) (2021)
- [Mapping the Internet: Modelling Entity Interactions in Complex Heterogeneous Networks](https://mlsanity.com/paper/2104.09650/) (2021)
- [NeuralFMU: Presenting a workflow for integrating hybrid NeuralODEs into real world applications](https://mlsanity.com/paper/2209.03933/) (2022)
- [Recursive Neural Programs: Variational Learning of Image Grammars and Part-Whole Hierarchies](https://mlsanity.com/paper/2206.08462/) (2022)
- [Physics informed machine learning with Smoothed Particle Hydrodynamics: Hierarchy of reduced Lagrangian models of turbulence](https://mlsanity.com/paper/2110.13311/) (2021)
- [Optimisation of Structured Neural Controller Based on Continuous-Time Policy Gradient](https://mlsanity.com/paper/2201.06262/) (2022)
- [RobustNeuralNetworks.jl: a Package for Machine Learning and Data-Driven Control with Certified Robustness](https://mlsanity.com/paper/2306.12612/) (2023)
- [Generalization vs. Memorization in Autoregressive Deep Learning: Or, Examining Temporal Decay of Gradient Coherence](https://mlsanity.com/paper/2509.00024/) (2025)
- [Distributionally Robust PAC-Bayesian Control](https://mlsanity.com/paper/2604.10588/) (2026)

## Abstract (arXiv)

This paper presents reverse-mode algorithmic differentiation (AD) based on source code transformation, in particular of the Static Single Assignment (SSA) form used by modern compilers. The approach can support control flow, nesting, mutation, recursion, data structures, higher-order functions, and other language constructs, and the output is given to an existing compiler to produce highly efficient differentiated code. Our implementation is a new AD tool for the Julia language, called Zygote, which presents high-level dynamic semantics while transparently compiling adjoint code under the hood. We discuss the benefits of this approach to both the usability and performance of AD tools.