# Smoothing Methods for Automatic Differentiation Across Conditional Branches

arXiv: https://arxiv.org/abs/2310.03585 · HTML: https://mlsanity.com/paper/2310.03585/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 66 references that are in the map.
- First posted on arXiv 5 Oct 2023.

## Builds on (most cited first, 23 of 23)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [The Loss Surfaces of Multilayer Networks](https://mlsanity.com/paper/1412.0233/) (2014)
- [Differentiable Rendering: A Survey](https://mlsanity.com/paper/2006.12057/) (2020)
- [Gradients without Backpropagation](https://mlsanity.com/paper/2202.08587/) (2022)
- [Programming with a Differentiable Forth Interpreter](https://mlsanity.com/paper/1605.06640/) (2016)
- [Learning Differentiable Programs with Admissible Neural Heuristics](https://mlsanity.com/paper/2007.12101/) (2020)
- [Differentiable Programs with Neural Libraries](https://mlsanity.com/paper/1611.02109/) (2016)
- [Automatic Differentiation of Programs with Discrete Randomness](https://mlsanity.com/paper/2210.08572/) (2022)
- [Bundled Gradients through Contact via Randomized Smoothing](https://mlsanity.com/paper/2109.05143/) (2021)
- [Gradient Sampling Methods for Nonsmooth Optimization](https://mlsanity.com/paper/1804.11003/) (2018)
- [NEUZZ: Efficient Fuzzing with Neural Program Smoothing](https://mlsanity.com/paper/1807.05620/) (2018)
- [TerpreT: A Probabilistic Programming Language for Program Induction](https://mlsanity.com/paper/1608.04428/) (2016)
- [ADEV: Sound Automatic Differentiation of Expected Values of Probabilistic Programs](https://mlsanity.com/paper/2212.06386/) (2022)
- [A Simple Differentiable Programming Language](https://mlsanity.com/paper/1911.04523/) (2019)
- [Learning with Differentiable Algorithms](https://mlsanity.com/paper/2209.00616/) (2022)
- [Safe Neurosymbolic Learning with Differentiable Symbolic Execution](https://mlsanity.com/paper/2203.07671/) (2022)
- [λS: Computable Semantics for Differentiable Programming with Higher-Order Functions and Datatypes](https://mlsanity.com/paper/2007.08017/) (2020)
- [ASTRO-DF: A Class of Adaptive Sampling Trust-Region Algorithms for Derivative-Free Stochastic Optimization](https://mlsanity.com/paper/1610.06506/) (2016)
- [Differentiable Functional Program Interpreters](https://mlsanity.com/paper/1611.01988/) (2016)
- [Neurosymbolic Programming for Science](https://mlsanity.com/paper/2210.05050/) (2022)
- [Differentiable Programming: Efficient Smoothing of Control-Flow-Induced Discontinuities](https://mlsanity.com/paper/2305.06692/) (2023)
- [AlgoNet: C^∞ Smooth Algorithmic Neural Networks](https://mlsanity.com/paper/1905.06886/) (2019)
- [Distribution Theoretic Semantics for Non-Smooth Differentiable Programming](https://mlsanity.com/paper/2207.05946/) (2022)

## Built on by (most cited first, 7 of 7)

- [Probabilistic Programming with Programmable Variational Inference](https://mlsanity.com/paper/2406.15742/) (2024)
- [Resource-efficient Inference with Foundation Model Programs](https://mlsanity.com/paper/2504.07247/) (2025)
- [Automatic Gradient Estimation for Calibrating Crowd Models with Discrete Decision Making](https://mlsanity.com/paper/2404.04678/) (2024)
- [Towards Learning Stochastic Population Models by Gradient Descent](https://mlsanity.com/paper/2404.07049/) (2024)
- [Dimensional Peeking for Low-Variance Gradients in Zeroth-Order Discrete Optimization via Simulation](https://mlsanity.com/paper/2602.00075/) (2026)
- [Compile Once, Differentiate Everywhere: A Differentiable Meta-Circular Interpreter](https://mlsanity.com/paper/2606.09930/) (2026)
- [Differentiate the Evaluator, Not the Program: An Efficient Runtime Representation for Neuro-Symbolic Learning](https://mlsanity.com/paper/2607.03574/) (2026)

## Abstract (arXiv)

Programs involving discontinuities introduced by control flow constructs such as conditional branches pose challenges to mathematical optimization methods that assume a degree of smoothness in the objective function's response surface. Smooth interpretation (SI) is a form of abstract interpretation that approximates the convolution of a program's output with a Gaussian kernel, thus smoothing its output in a principled manner. Here, we combine SI with automatic differentiation (AD) to efficiently compute gradients of smoothed programs. In contrast to AD across a regular program execution, these gradients also capture the effects of alternative control flow paths. The combination of SI with AD enables the direct gradient-based parameter synthesis for branching programs, allowing for instance the calibration of simulation models or their combination with neural network models in machine learning pipelines. We detail the effects of the approximations made for tractability in SI and propose a novel Monte Carlo estimator that avoids the underlying assumptions by estimating the smoothed programs' gradients through a combination of AD and sampling. Using DiscoGrad, our tool for automatically translating simple C++ programs to a smooth differentiable form, we perform an extensive evaluation. We compare the combination of SI with AD and our Monte Carlo estimator to existing gradient-free and stochastic methods on four non-trivial and originally discontinuous problems ranging from classical simulation-based optimization to neural network-driven control. While the optimization progress with the SI-based estimator depends on the complexity of the program's control flow, our Monte Carlo estimator is competitive in all problems, exhibiting the fastest convergence by a substantial margin in our highest-dimensional problem.