# Quantum Logic Gate Synthesis as a Markov Decision Process

arXiv: https://arxiv.org/abs/1912.12002 · HTML: https://mlsanity.com/paper/1912.12002/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Dec 2019.

## Built on by (most cited first, 6 of 6)

- [Reinforcement learning for optimization of variational quantum circuit architectures](https://mlsanity.com/paper/2103.16089/) (2021)
- [Efficient and practical quantum compiler towards multi-qubit systems with deep reinforcement learning](https://mlsanity.com/paper/2204.06904/) (2022)
- [Weighted Quantum Channel Compiling through Proximal Policy Optimization](https://mlsanity.com/paper/2111.02426/) (2021)
- [Compilation of product-formula Hamiltonian simulation via reinforcement learning](https://mlsanity.com/paper/2311.04285/) (2023)
- [Dynamic Inhomogeneous Quantum Resource Scheduling with Reinforcement Learning](https://mlsanity.com/paper/2405.16380/) (2024)
- [Model-free Distortion Canceling and Control of Quantum Devices](https://mlsanity.com/paper/2407.09877/) (2024)

## Abstract (arXiv)

Reinforcement learning has witnessed recent applications to a variety of tasks in quantum programming. The underlying assumption is that those tasks could be modeled as Markov Decision Processes (MDPs). Here, we investigate the feasibility of this assumption by exploring its consequences for two fundamental tasks in quantum programming: state preparation and gate compilation. By forming discrete MDPs, focusing exclusively on the single-qubit case (both with and without noise), we solve for the optimal policy exactly through policy iteration. We find optimal paths that correspond to the shortest possible sequence of gates to prepare a state, or compile a gate, up to some target accuracy. As an example, we find sequences of H and T gates with length as small as 11 producing ~ 99% fidelity for states of the form (HT)n |0rangle with values as large as n=1010. In the presence of gate noise, we demonstrate how the optimal policy adapts to the effects of noisy gates in order to achieve a higher state fidelity. Our work shows that one can meaningfully impose a discrete, stochastic and Markovian nature to a continuous, deterministic and non-Markovian quantum evolution, and provides theoretical insight into why reinforcement learning may be successfully used to find optimally short gate sequences in quantum programming.