# A Differentiable Programming System to Bridge Machine Learning and Scientific Computing

arXiv: https://arxiv.org/abs/1907.07587 · HTML: https://mlsanity.com/paper/1907.07587/

## Key facts

- Cited by 83 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Jul 2019.

## Built on by (most cited first, 30 of 83)

- [Neural SDEs as Infinite-Dimensional GANs](https://mlsanity.com/paper/2102.03657/) (2021)
- [E^2VPT: An Effective and Efficient Approach for Visual Prompt Tuning](https://mlsanity.com/paper/2307.13770/) (2023)
- [Physics-Informed Machine Learning for Modeling and Control of Dynamical Systems](https://mlsanity.com/paper/2306.13867/) (2023)
- [Learning neural network potentials from experimental data via Differentiable Trajectory Reweighting](https://mlsanity.com/paper/2106.01138/) (2021)
- [Physics-Guided, Physics-Informed, and Physics-Encoded Neural Networks in Scientific Computing](https://mlsanity.com/paper/2211.07377/) (2022)
- [Differentiable Programming for Earth System Modeling](https://mlsanity.com/paper/2208.13825/) (2022)
- [Training Efficient Controllers via Analytic Policy Gradient](https://mlsanity.com/paper/2209.13052/) (2022)
- [Flexible Differentiable Optimization via Model Transformations](https://mlsanity.com/paper/2206.06135/) (2022)
- [Neural Ordinary Differential Equations for Nonlinear System Identification](https://mlsanity.com/paper/2203.00120/) (2022)
- [Sensitivity-Constrained Fourier Neural Operators for Forward and Inverse Problems in Parametric Differential Equations](https://mlsanity.com/paper/2505.08740/) (2025)
- [Quantum Neural Network Classifiers: A Tutorial](https://mlsanity.com/paper/2206.02806/) (2022)
- [NURBS-Diff: A Differentiable Programming Module for NURBS](https://mlsanity.com/paper/2104.14547/) (2021)
- [j-Wave: An open-source differentiable wave simulator](https://mlsanity.com/paper/2207.01499/) (2022)
- [Quantum HyperNetworks: Training Binary Neural Networks in Quantum Superposition](https://mlsanity.com/paper/2301.08292/) (2023)
- [Physics-informed machine learning with differentiable programming for heterogeneous underground reservoir pressure management](https://mlsanity.com/paper/2206.10718/) (2022)
- [Learned multiphysics inversion with differentiable programming and machine learning](https://mlsanity.com/paper/2304.05592/) (2023)
- [Differentiable Visual Computing for Inverse Problems and Machine Learning](https://mlsanity.com/paper/2312.04574/) (2023)
- [InvertibleNetworks.jl: A Julia package for scalable normalizing flows](https://mlsanity.com/paper/2312.13480/) (2023)
- [Fixes That Fail: Self-Defeating Improvements in Machine-Learning Systems](https://mlsanity.com/paper/2103.11766/) (2021)
- [Robot Program Parameter Inference via Differentiable Shadow Program Inversion](https://mlsanity.com/paper/2103.14452/) (2021)
- [Symplectic Autoencoders for Model Reduction of Hamiltonian Systems](https://mlsanity.com/paper/2312.10004/) (2023)
- [DiffHybrid-UQ: Uncertainty Quantification for Differentiable Hybrid Neural Modeling](https://mlsanity.com/paper/2401.00161/) (2023)
- [JAX-MPM: A Learning-Augmented Differentiable Meshfree Framework for GPU-Accelerated Lagrangian Simulation and Geophysical Inverse Modeling](https://mlsanity.com/paper/2507.04192/) (2025)
- [Control of Stochastic Quantum Dynamics by Differentiable Programming](https://mlsanity.com/paper/2101.01190/) (2021)
- [Individual Survival Curves with Conditional Normalizing Flows](https://mlsanity.com/paper/2107.12825/) (2021)
- [Differentiable Spline Approximations](https://mlsanity.com/paper/2110.01532/) (2021)
- [Learning Stochastic Parametric Differentiable Predictive Control Policies](https://mlsanity.com/paper/2203.01447/) (2022)
- [Differentiable modeling to unify machine learning and physical models and advance Geosciences](https://mlsanity.com/paper/2301.04027/) (2023)
- [Neural-Integrated Meshfree (NIM) Method: A differentiable programming-based hybrid solver for computational mechanics](https://mlsanity.com/paper/2311.12915/) (2023)
- [Sparser, Better, Faster, Stronger: Sparsity Detection for Efficient Automatic Differentiation](https://mlsanity.com/paper/2501.17737/) (2025)

## Abstract (arXiv)

Scientific computing is increasingly incorporating the advancements in machine learning and the ability to work with large amounts of data. At the same time, machine learning models are becoming increasingly sophisticated and exhibit many features often seen in scientific computing, stressing the capabilities of machine learning frameworks. Just as the disciplines of scientific computing and machine learning have shared common underlying infrastructure in the form of numerical linear algebra, we now have the opportunity to further share new computational infrastructure, and thus ideas, in the form of Differentiable Programming. We describe Zygote, a Differentiable Programming system that is able to take gradients of general program structures. We implement this system in the Julia programming language. Our system supports almost all language constructs (control flow, recursion, mutation, etc.) and compiles high-performance code without requiring any user intervention or refactoring to stage computations. This enables an expressive programming model for deep learning, but more importantly, it enables us to incorporate a large ecosystem of libraries in our models in a straightforward way. We discuss our approach to automatic differentiation, including its support for advanced techniques such as mixed-mode, complex and checkpointed differentiation, and present several examples of differentiating programs.