# A Reinforcement Learning Environment for Directed Quantum Circuit Synthesis

arXiv: https://arxiv.org/abs/2401.07054 · HTML: https://mlsanity.com/paper/2401.07054/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Placed in Quantum reinforcement learning, one of 15 directions in Deep reinforcement learning methods.
- Builds on 6 of its 29 references that are in the map.
- First posted on arXiv 13 Jan 2024.

## Placed in

- Deep reinforcement learning methods / [Quantum reinforcement learning](https://mlsanity.com/deeprl/quantum-reinforcement-learning/)

## Builds on (most cited first, 6 of 6)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Asynchronous Methods for Deep Reinforcement Learning](https://mlsanity.com/paper/1602.01783/) (2016)
- [Variational Quantum Algorithms](https://mlsanity.com/paper/2012.09265/) (2020)
- [Reinforcement learning for optimization of variational quantum circuit architectures](https://mlsanity.com/paper/2103.16089/) (2021)
- [Quantum Circuit Design Search](https://mlsanity.com/paper/2012.04046/) (2020)
- [Quarl: A Learning-Based Quantum Circuit Optimizer](https://mlsanity.com/paper/2307.10120/) (2023)

## Built on by (most cited first, 10 of 10)

- [Evaluating Mutation Techniques in Genetic Algorithm-Based Quantum Circuit Synthesis](https://mlsanity.com/paper/2504.06413/) (2025)
- [Reinforcement Learning to Disentangle Multiqubit Quantum States from Partial Observations](https://mlsanity.com/paper/2406.07884/) (2024)
- [Enhancing Quantum Memory Lifetime with Measurement-Free Local Error Correction and Reinforcement Learning](https://mlsanity.com/paper/2408.09524/) (2024)
- [Quantum Machine Learning for Optimizing Entanglement Distribution in Quantum Sensor Circuits](https://mlsanity.com/paper/2508.21252/) (2025)
- [Reinforcement Learning for Optimizing Large Qubit Array based Quantum Sensor Circuits](https://mlsanity.com/paper/2508.21253/) (2025)
- [Quantum Architecture Search for Solving Quantum Machine Learning Tasks](https://mlsanity.com/paper/2509.11198/) (2025)
- [Reinforcement Learning for Adaptive Composition of Quantum Circuit Optimisation Passes](https://mlsanity.com/paper/2601.21629/) (2026)
- [Reinforcement Learning for Parameterized Quantum State Preparation: A Comparative Study](https://mlsanity.com/paper/2602.16523/) (2026)
- [Investigation of Automated Design of Quantum Circuits for Imaginary Time Evolution Methods Using Deep Reinforcement Learning](https://mlsanity.com/paper/2604.07951/) (2026)
- [Generative Quantum Data Embeddings for Supervised Learning](https://mlsanity.com/paper/2605.30866/) (2026)

## Abstract (arXiv)

With recent advancements in quantum computing technology, optimizing quantum circuits and ensuring reliable quantum state preparation have become increasingly vital. Traditional methods often demand extensive expertise and manual calculations, posing challenges as quantum circuits grow in qubit- and gate-count. Therefore, harnessing machine learning techniques to handle the growing variety of gate-to-qubit combinations is a promising approach. In this work, we introduce a comprehensive reinforcement learning environment for quantum circuit synthesis, where circuits are constructed utilizing gates from the the Clifford+T gate set to prepare specific target states. Our experiments focus on exploring the relationship between the depth of synthesized quantum circuits and the circuit depths used for target initialization, as well as qubit count. We organize the environment configurations into multiple evaluation levels and include a range of well-known quantum states for benchmarking purposes. We also lay baselines for evaluating the environment using Proximal Policy Optimization. By applying the trained agents to benchmark tests, we demonstrated their ability to reliably design minimal quantum circuits for a selection of 2-qubit Bell states.