# Control of Stochastic Quantum Dynamics by Differentiable Programming

arXiv: https://arxiv.org/abs/2101.01190 · HTML: https://mlsanity.com/paper/2101.01190/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 78 references that are in the map.
- First posted on arXiv 4 Jan 2021.

## Builds on (most cited first, 16 of 16)

- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Automatic differentiation in machine learning: a survey](https://mlsanity.com/paper/1502.05767/) (2015)
- [DGM: A deep learning algorithm for solving partial differential equations](https://mlsanity.com/paper/1708.07469/) (2017)
- [Universal Differential Equations for Scientific Machine Learning](https://mlsanity.com/paper/2001.04385/) (2020)
- [Discovering Symbolic Models from Deep Learning with Inductive Biases](https://mlsanity.com/paper/2006.11287/) (2020)
- [Scalable Gradients for Stochastic Differential Equations](https://mlsanity.com/paper/2001.01328/) (2020)
- [Neural Jump Stochastic Differential Equations](https://mlsanity.com/paper/1905.10403/) (2019)
- [Sparse identification of nonlinear dynamics for model predictive control in the low-data limit](https://mlsanity.com/paper/1711.05501/) (2017)
- [A Differentiable Programming System to Bridge Machine Learning and Scientific Computing](https://mlsanity.com/paper/1907.07587/) (2019)
- [Sparse learning of stochastic dynamic equations](https://mlsanity.com/paper/1712.02432/) (2017)
- [Universal Quantum Control through Deep Reinforcement Learning](https://mlsanity.com/paper/1803.01857/) (2018)
- [DiffEqFlux.jl - A Julia Library for Neural Differential Equations](https://mlsanity.com/paper/1902.02376/) (2019)
- [The Koopman Expectation: An Operator Theoretic Method for Efficient Analysis and Optimization of Uncertain Hybrid Dynamical Systems](https://mlsanity.com/paper/2008.08737/) (2020)
- [Automatic Backward Filtering Forward Guiding for Markov processes and graphical models](https://mlsanity.com/paper/2010.03509/) (2020)
- [Analysis and Control of Stochastic Systems using Semidefinite Programming over Moments](https://mlsanity.com/paper/1702.00422/) (2017)

## Built on by (most cited first, 4 of 4)

- [Near-optimal control of dynamical systems with neural ordinary differential equations](https://mlsanity.com/paper/2206.11120/) (2022)
- [AbstractDifferentiation.jl: Backend-Agnostic Differentiable Programming in Julia](https://mlsanity.com/paper/2109.12449/) (2021)
- [Sample-efficient Model-based Reinforcement Learning for Quantum Control](https://mlsanity.com/paper/2304.09718/) (2023)
- [Model-free optimization of power/efficiency tradeoffs in quantum thermal machines using reinforcement learning](https://mlsanity.com/paper/2204.04785/) (2022)

## Abstract (arXiv)

Control of the stochastic dynamics of a quantum system is indispensable in fields such as quantum information processing and metrology. However, there is no general ready-made approach to the design of efficient control strategies. Here, we propose a framework for the automated design of control schemes based on differentiable programming (partial P). We apply this approach to the state preparation and stabilization of a qubit subjected to homodyne detection. To this end, we formulate the control task as an optimization problem where the loss function quantifies the distance from the target state, and we employ neural networks (NNs) as controllers. The system's time evolution is governed by a stochastic differential equation (SDE). To implement efficient training, we backpropagate the gradient information from the loss function through the SDE solver using adjoint sensitivity methods. As a first example, we feed the quantum state to the controller and focus on different methods of obtaining gradients. As a second example, we directly feed the homodyne detection signal to the controller. The instantaneous value of the homodyne current contains only very limited information on the actual state of the system, masked by unavoidable photon-number fluctuations. Despite the resulting poor signal-to-noise ratio, we can train our controller to prepare and stabilize the qubit to a target state with a mean fidelity of around 85%. We also compare the solutions found by the NN to a hand-crafted control strategy.