# Optimizing the non-Clifford-count in unitary synthesis using Reinforcement Learning

arXiv: https://arxiv.org/abs/2509.21709 · HTML: https://mlsanity.com/paper/2509.21709/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in Quantum reinforcement learning, one of 15 directions in Deep reinforcement learning methods.
- Builds on 9 of its 94 references that are in the map.
- First posted on arXiv 26 Sep 2025.

## Placed in

- Deep reinforcement learning methods / [Quantum reinforcement learning](https://mlsanity.com/deeprl/quantum-reinforcement-learning/)

## Builds on (most cited first, 9 of 9)

- [Molecular Transformer - A Model for Uncertainty-Calibrated Chemical Reaction Prediction](https://mlsanity.com/paper/1811.02633/) (2018)
- [Quantum Circuit Optimization with AlphaTensor](https://mlsanity.com/paper/2402.14396/) (2024)
- [Quantum circuit synthesis with diffusion models](https://mlsanity.com/paper/2311.02041/) (2023)
- [Practical and efficient quantum circuit synthesis and transpiling with Reinforcement Learning](https://mlsanity.com/paper/2405.13196/) (2024)
- [Variational quantum compiling with double Q-learning](https://mlsanity.com/paper/2103.11611/) (2021)
- [Optimizing Quantum Variational Circuits with Deep Reinforcement Learning](https://mlsanity.com/paper/2109.03188/) (2021)
- [AutoQC: Automated Synthesis of Quantum Circuits Using Neural Network](https://mlsanity.com/paper/2210.02766/) (2022)
- [Pauli Network Circuit Synthesis with Reinforcement Learning](https://mlsanity.com/paper/2503.14448/) (2025)
- [Recursive Methods for Synthesizing Permutations on Limited-Connectivity Quantum Computers](https://mlsanity.com/paper/2207.06199/) (2022)

## Built on by (most cited first, 5 of 5)

- [Beyond Reinforcement Learning: Fast and Scalable Quantum Circuit Synthesis](https://mlsanity.com/paper/2602.15146/) (2026)
- [Quantum Circuit Pre-Synthesis: Learning Local Edits to Reduce T-count](https://mlsanity.com/paper/2601.19738/) (2026)
- [Aligning Quantum Operators with Large Language Models](https://mlsanity.com/paper/2606.13811/) (2026)
- [Classical State Preparation for Variational Quantum Algorithms via Reinforcement Learning](https://mlsanity.com/paper/2605.23138/) (2026)
- [AlphaClifford: Efficient Clifford Synthesis and Transpilation with Model-based RL](https://mlsanity.com/paper/2608.18946/) (2026)

## Abstract (arXiv)

In this paper we study the potential of using reinforcement learning (RL) in order to synthesize quantum circuits, while optimizing the T-count and CS-count, of unitaries that are exactly implementable by the Clifford+T and Clifford+CS gate sets, respectively. We have designed our RL framework to work with channel representation of unitaries, that enables us to perform matrix operations efficiently, using integers only. We have also incorporated pruning heuristics and a canonicalization of operators, in order to reduce the search complexity. As a result, compared to previous works, we are able to implement significantly larger unitaries, in less time, with much better success rate and improvement factor. Our results for Clifford+T synthesis on two qubit unitaries achieve close-to-optimal decompositions for up to 100 T gates, 5 times more than previous RL algorithms and to the best of our knowledge, the largest instances achieved with any method to date. Our RL algorithm is able to recover previously-known optimal linear complexity algorithm for T-count-optimal decomposition of 1 qubit unitaries. We illustrate significant reduction in the asymptotic T-count estimate of important primitives like controlled cyclic shift (43%), controlled adder (14.3%) and multiplier (14%), without adding any extra ancilla. For 2-qubit Clifford+CS unitaries, our algorithm achieves a linear complexity, something that could only be accomplished by a previous algorithm using SO(6) representation.