# Model-free Distortion Canceling and Control of Quantum Devices

arXiv: https://arxiv.org/abs/2407.09877 · HTML: https://mlsanity.com/paper/2407.09877/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Quantum reinforcement learning, one of 15 directions in Deep reinforcement learning methods.
- Builds on 21 of its 111 references that are in the map.
- First posted on arXiv 13 Jul 2024.

## Placed in

- Deep reinforcement learning methods / [Quantum reinforcement learning](https://mlsanity.com/deeprl/quantum-reinforcement-learning/)

## Builds on (most cited first, 21 of 21)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [A Brief Survey of Deep Reinforcement Learning](https://mlsanity.com/paper/1708.05866/) (2017)
- [Universal Quantum Control through Deep Reinforcement Learning](https://mlsanity.com/paper/1803.01857/) (2018)
- [Optimizing Quantum Error Correction Codes with Reinforcement Learning](https://mlsanity.com/paper/1812.08451/) (2018)
- [Ridge Regularization: an Essential Concept in Data Science](https://mlsanity.com/paper/2006.00371/) (2020)
- [Policy Gradient based Quantum Approximate Optimization Algorithm](https://mlsanity.com/paper/2002.01068/) (2020)
- [Deep Reinforcement Learning Control of Quantum Cartpoles](https://mlsanity.com/paper/1910.09200/) (2019)
- [Optimizing Quantum Variational Circuits with Deep Reinforcement Learning](https://mlsanity.com/paper/2109.03188/) (2021)
- [Quantum Logic Gate Synthesis as a Markov Decision Process](https://mlsanity.com/paper/1912.12002/) (2019)
- [Experimental graybox quantum system identification and control](https://mlsanity.com/paper/2206.12201/) (2022)
- [Approximation of quantum control correction scheme using deep neural networks](https://mlsanity.com/paper/1803.05193/) (2018)
- [Modern Deep Reinforcement Learning Algorithms](https://mlsanity.com/paper/1906.10025/) (2019)
- [Quantum Geometric Machine Learning for Quantum Circuits and Control](https://mlsanity.com/paper/2006.11332/) (2020)
- [Noise Detection with Spectator Qubits and Quantum Feature Engineering](https://mlsanity.com/paper/2103.13018/) (2021)
- [Multi-Axis Control of a Qubit in the Presence of Unknown Non-Markovian Quantum Noise](https://mlsanity.com/paper/2208.03058/) (2022)
- [Taking gradients through experiments: LSTMs and memory proximal policy optimization for black-box quantum control](https://mlsanity.com/paper/1802.04063/) (2018)
- [Quantum control in the presence of strongly coupled non-Markovian noise](https://mlsanity.com/paper/2404.19251/) (2024)
- [Real-time calibration of coherent-state receivers: learning by trial and error](https://mlsanity.com/paper/2001.10283/) (2020)
- [Auxiliary Task-based Deep Reinforcement Learning for Quantum Control](https://mlsanity.com/paper/2302.14312/) (2023)

## Abstract (arXiv)

Quantum devices need precise control to achieve their full capability. In this work, we address the problem of controlling closed quantum systems, tackling two main issues. First, in practice the control signals are usually subject to unknown classical distortions that could arise from the device fabrication, material properties and/or instruments generating those signals. Second, in most cases modeling the system is very difficult or not even viable due to uncertainties in the relations between some variables and inaccessibility to some measurements inside the system. In this paper, we introduce a general model-free control approach based on deep reinforcement learning (DRL), that can work for any closed quantum system. We train a deep neural network (NN), using the REINFORCE policy gradient algorithm to control the state probability distribution of a closed quantum system as it evolves, and drive it to different target distributions. We present a novel controller architecture that comprises multiple NNs. This enables accommodating as many different target state distributions as desired, without increasing the complexity of the NN or its training process. The used DRL algorithm works whether the control problem can be modeled as a Markov decision process (MDP) or a partially observed MDP. Our method is valid whether the control signals are discrete- or continuous-valued. We verified our method through numerical simulations based on a photonic waveguide array chip. We trained a controller to generate sequences of different target output distributions of the chip with fidelity higher than 99%, where the controller showed superior performance in canceling the classical signal distortions.