# Differentiable Agent-Based Simulation for Gradient-Guided Simulation-Based Optimization

arXiv: https://arxiv.org/abs/2103.12476 · HTML: https://mlsanity.com/paper/2103.12476/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 71 references that are in the map.
- First posted on arXiv 23 Mar 2021.

## Builds on (most cited first, 14 of 14)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [An overview of gradient descent optimization algorithms](https://mlsanity.com/paper/1609.04747/) (2016)
- [Automatic differentiation in machine learning: a survey](https://mlsanity.com/paper/1502.05767/) (2015)
- [DiffTaichi: Differentiable Programming for Physical Simulation](https://mlsanity.com/paper/1910.00935/) (2019)
- [Fast Differentiable Sorting and Ranking](https://mlsanity.com/paper/2002.08871/) (2020)
- [Value Iteration Networks](https://mlsanity.com/paper/1602.02867/) (2016)
- [Differentiable Ranks and Sorting using Optimal Transport](https://mlsanity.com/paper/1905.11885/) (2019)
- [Stochastic Optimization of Sorting Networks via Continuous Relaxations](https://mlsanity.com/paper/1903.08850/) (2019)
- [A Differentiable Physics Engine for Deep Learning in Robotics](https://mlsanity.com/paper/1611.01652/) (2016)
- [Accurate Calibration of Agent-based Epidemiological Models with Neural Network Surrogates](https://mlsanity.com/paper/2010.06558/) (2020)
- [Efficient Differentiable Programming in a Functional Array-Processing Language](https://mlsanity.com/paper/1806.02136/) (2018)
- [Differentiable programming and its applications to dynamical systems](https://mlsanity.com/paper/1912.08168/) (2019)
- [A Survey on Agent-based Simulation using Hardware Accelerators](https://mlsanity.com/paper/1807.01014/) (2018)
- [ensmallen: a flexible C++ library for efficient function optimization](https://mlsanity.com/paper/1810.09361/) (2018)

## Built on by (most cited first, 8 of 8)

- [Differentiable Agent-based Epidemiology](https://mlsanity.com/paper/2207.09714/) (2022)
- [Differentiable Hybrid Traffic Simulation](https://mlsanity.com/paper/2210.08046/) (2022)
- [Traffic-Aware Autonomous Driving with Differentiable Traffic Simulation](https://mlsanity.com/paper/2210.03772/) (2022)
- [Variational Inference of Parameters in Opinion Dynamics Models](https://mlsanity.com/paper/2403.05358/) (2024)
- [Some challenges of calibrating differentiable agent-based models](https://mlsanity.com/paper/2307.01085/) (2023)
- [Gradient-based Trajectory Optimization with Parallelized Differentiable Traffic Simulation](https://mlsanity.com/paper/2412.16750/) (2024)
- [Ultra-fast Traffic Nowcasting and Control via Differentiable Agent-based Simulation](https://mlsanity.com/paper/2603.25068/) (2026)
- [Learning Collective Dynamics with Differentiable Gaussian Representations](https://mlsanity.com/paper/2609.28405/) (2026)

## Abstract (arXiv)

Simulation-based optimization using agent-based models is typically carried out under the assumption that the gradient describing the sensitivity of the simulation output to the input cannot be evaluated directly. To still apply gradient-based optimization methods, which efficiently steer the optimization towards a local optimum, gradient estimation methods can be employed. However, many simulation runs are needed to obtain accurate estimates if the input dimension is large. Automatic differentiation (AD) is a family of techniques to compute gradients of general programs directly. Here, we explore the use of AD in the context of time-driven agent-based simulations. By substituting common discrete model elements such as conditional branching with smooth approximations, we obtain gradient information across discontinuities in the model logic. On the example of microscopic traffic models and an epidemics model, we study the fidelity and overhead of the differentiable models, as well as the convergence speed and solution quality achieved by gradient-based optimization compared to gradient-free methods. In traffic signal timing optimization problems with high input dimension, the gradient-based methods exhibit substantially superior performance. Finally, we demonstrate that the approach enables gradient-based training of neural network-controlled simulation entities embedded in the model logic.