mlsanityData to 7 Oct 2026

The Elements of Differentiable Programming

arXiv 2403.14606 ↗First posted 21 Mar 2024cs.LG cs.AI cs.PLcited by 41

Not on the map. Why?

Abstract (arXiv)

Artificial intelligence has recently experienced remarkable advances, fueled by large models, vast datasets, accelerated hardware, and, last but not least, the transformative power of differentiable programming. This new programming paradigm enables end-to-end differentiation of complex computer programs (including those with control flows and data structures), making gradient-based optimization of program parameters possible. As an emerging paradigm, differentiable programming builds upon several areas of computer science and applied mathematics, including automatic differentiation, graphical models, optimization and statistics. This book presents a comprehensive review of the fundamental concepts useful for differentiable programming. We adopt two main perspectives, that of optimization and that of probability, with clear analogies between the two. Differentiable programming is not merely the differentiation of programs, but also the thoughtful design of programs intended for differentiation. By making programs differentiable, we inherently introduce probability distributions over their execution, providing a means to quantify the uncertainty associated with program outputs.

Key facts. Cited by 41 papers in the map (data through 7 Oct 2026). First posted on arXiv 21 Mar 2024.

Builds on

0 of 0 references.

No references here yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 6; 2025: 23; 2026: 12.

41 papers.

Sort:

By area:

  1. 2025Distillation Scaling Lawscited by 59
  2. 2025Uncertainty Quantification with the Empirical Neural Tangent Kernelcited by 11
  3. 2024Differentiation Through Black-Box Quadratic Programming Solverscited by 10
  4. 2025Loss Functions and Operators Generated by f-Divergencescited by 10
  5. 2024Contrasting Multiple Representations with the Multi-Marginal Matching Gapcited by 8
  6. 2024Stepping on the Edge: Curvature Aware Learning Rate Tunerscited by 8
  7. 2025End-to-End Probabilistic Framework for Learning with Hard Constraintscited by 8
  8. 2025Structured Reinforcement Learning for Combinatorial Decision-Makingcited by 7
  9. 2025Autoregressive Language Models are Secretly Energy-Based Models: Insights into the Lookahead Capabilities of Next-Token Predictioncited by 7
  10. 2025Learning long range dependencies through time reversal symmetry breakingcited by 6
  11. 2025Sparser, Better, Faster, Stronger: Sparsity Detection for Efficient Automatic Differentiationcited by 4
  12. 2025DDEQs: Distributional Deep Equilibrium Models through Wasserstein Gradient Flowscited by 4
29 more
  1. 2025Scalable Meta-Learning via Mixed-Mode Differentiationcited by 4
  2. 2025Joint Learning of Energy-based Models and their Partition Functioncited by 3
  3. 2025PRDP: Progressively Refined Differentiable Physicscited by 3
  4. 2026Combinatorial Optimization Augmented Machine Learningcited by 3
  5. 2024Algorithmic Language Models with Neurally Compiled Librariescited by 2
  6. 2025Establishing Linear Surrogate Regret Bounds for Convex Smooth Losses via Convolutional Fenchel-Young Lossescited by 2
  7. 2025Learning with Local Search MCMC Layerscited by 2
  8. 2025ConStellaration: A dataset of QI-like stellarator plasma boundaries and optimization benchmarkscited by 2
  9. 2025Per-example gradients: a new frontier for understanding and improving optimizerscited by 2
  10. 2025Neural non-canonical Hamiltonian dynamics for long-time simulationscited by 2
  11. 2026TurboMPC: Fast, Scalable, and Differentiable Model Predictive Control on the GPUcited by 2
  12. 2024Gradients of Functions of Large Matricescited by 1
  13. 2024Shadow Program Inversion with Differentiable Planning: A Framework for Unified Robot Program Parameter and Trajectory Optimizationcited by 1
  14. 2025Implicit Neural Differential Model for Spatiotemporal Dynamicscited by 1
  15. 2025Reinforced Embodied Active Defense: Exploiting Adaptive Interaction for Robust Visual Perception in Adversarial 3D Environmentscited by 1
  16. 2025Matrix-Free Least Squares Solvers: Values, Gradients, and What to Do With Themcited by 1
  17. 2025First-ish Order Methods: Hessian-aware Scalings of Gradient Descentcited by 0
  18. 2025NNN: Next-Generation Neural Networks for Marketing Measurementcited by 0
  19. 2025Learning-Augmented Power System Operations: A Unified Optimization Viewcited by 0
  20. 2026Distributional Computational Graphs: Error Boundscited by 0
  21. 2026Embedding interpretable ℓ1-regression into neural networks for uncovering temporal structure in cell imagingcited by 0
  22. 2026A Study on Hidden Layer Distillation for Large Language Model Pre-Trainingcited by 0
  23. 2026Regularized Large Neighborhood Searchcited by 0
  24. 2026Integrating Mechanistic and Data-Driven Models for Neurological Disorders through Differentiable Programmingcited by 0
  25. 2026Wall Shear Stress Reconstruction from Concentration: Differentiable Physics and Physics-Informed Neural Networkscited by 0
  26. 2026DIFFRACT: Neuralized Utility Maximization for Wireless Networks by Differentiable Programmingcited by 0
  27. 2026Difference-of-Convex Regularization for Graph Learning by Differentiable Programmingcited by 0
  28. 2026Graph Matching Relaxations and Amortization for Supervised Graph Predictioncited by 0
  29. 2026asdex: Automatic Sparse Differentiation in JAXcited by 0