# Auxiliary Task-based Deep Reinforcement Learning for Quantum Control

arXiv: https://arxiv.org/abs/2302.14312 · HTML: https://mlsanity.com/paper/2302.14312/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 85 references that are in the map.
- First posted on arXiv 28 Feb 2023.

## Builds on (most cited first, 24 of 24)

- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Hindsight Experience Replay](https://mlsanity.com/paper/1707.01495/) (2017)
- [Deep Learning in Neural Networks: An Overview](https://mlsanity.com/paper/1404.7828/) (2014)
- [Reinforcement Learning with Unsupervised Auxiliary Tasks](https://mlsanity.com/paper/1611.05397/) (2016)
- [Learning to Navigate in Complex Environments](https://mlsanity.com/paper/1611.03673/) (2016)
- [Mastering the Game of Stratego with Model-Free Multiagent Reinforcement Learning](https://mlsanity.com/paper/2206.15378/) (2022)
- [Learning to Navigate in Cities Without a Map](https://mlsanity.com/paper/1804.00168/) (2018)
- [Learning Sparse Sharing Architectures for Multiple Tasks](https://mlsanity.com/paper/1911.05034/) (2019)
- [Quantum reinforcement learning in continuous action space](https://mlsanity.com/paper/2012.10711/) (2020)
- [Deep Reinforcement Learning with Quantum-inspired Experience Replay](https://mlsanity.com/paper/2101.02034/) (2021)
- [Quantum Language Model with Entanglement Embedding for Question Answering](https://mlsanity.com/paper/2008.09943/) (2020)
- [Asynchronous Episodic Deep Deterministic Policy Gradient: Towards Continuous Control in Computationally Complex Environments](https://mlsanity.com/paper/1903.00827/) (2019)
- [Admissible Policy Teaching through Reward Design](https://mlsanity.com/paper/2201.02185/) (2022)
- [Fidelity-based Probabilistic Q-learning for Control of Quantum Systems](https://mlsanity.com/paper/1806.03145/) (2018)
- [Learning of Long-Horizon Sparse-Reward Robotic Manipulator Tasks with Base Controllers](https://mlsanity.com/paper/2011.12105/) (2020)
- [Learning-based Quantum Robust Control: Algorithm, Applications and Experiments](https://mlsanity.com/paper/1702.03946/) (2017)
- [Curriculum-based Deep Reinforcement Learning for Quantum Control](https://mlsanity.com/paper/2012.15427/) (2020)
- [Model-free Quantum Gate Design and Calibration using Deep Reinforcement Learning](https://mlsanity.com/paper/2302.02371/) (2023)
- [Physics-Informed Quantum Communication Networks: A Vision Towards the Quantum Internet](https://mlsanity.com/paper/2204.09233/) (2022)
- [Policy Gradient Approach to Compilation of Variational Quantum Circuits](https://mlsanity.com/paper/2111.10227/) (2021)
- [Quantum estimation, control and learning: opportunities and challenges](https://mlsanity.com/paper/2201.05835/) (2022)
- [Rapid Lyapunov control of finite-dimensional quantum systems](https://mlsanity.com/paper/1512.02545/) (2015)
- [Finite-time stabilization control of quantum systems](https://mlsanity.com/paper/1910.00208/) (2019)

## Built on by (most cited first, 2 of 2)

- [Tomography of Quantum States from Structured Measurements via quantum-aware transformer](https://mlsanity.com/paper/2305.05433/) (2023)
- [Model-free Distortion Canceling and Control of Quantum Devices](https://mlsanity.com/paper/2407.09877/) (2024)

## Abstract (arXiv)

Due to its property of not requiring prior knowledge of the environment, reinforcement learning has significant potential for quantum control problems. In this work, we investigate the effectiveness of continuous control policies based on deep deterministic policy gradient. To solve the sparse reward signal in quantum learning control problems, we propose an auxiliary task-based deep reinforcement learning (AT-DRL) for quantum control. In particular, we first design a guided reward function based on the fidelity of quantum states that enables incremental fidelity improvement. Then, we introduce the concept of an auxiliary task whose network shares parameters with the main network to predict the reward provided by the environment (called the main task). The auxiliary task learns synchronously with the main task, allowing one to select the most relevant features of the environment, thus aiding the agent in comprehending how to achieve the desired state. The numerical simulations demonstrate that the proposed AT-DRL can provide a solution to the sparse reward in quantum systems, and has great potential in designing control pulses that achieve efficient quantum state preparation.