# AlphaClifford: Efficient Clifford Synthesis and Transpilation with Model-based RL

arXiv: https://arxiv.org/abs/2608.18946 · HTML: https://mlsanity.com/paper/2608.18946/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 15 of its 55 references that are in the map.
- First posted on arXiv 19 Aug 2026.

## Builds on (most cited first, 15 of 15)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Quantum Circuit Optimization with AlphaTensor](https://mlsanity.com/paper/2402.14396/) (2024)
- [Practical and efficient quantum circuit synthesis and transpiling with Reinforcement Learning](https://mlsanity.com/paper/2405.13196/) (2024)
- [Quantum Circuit Synthesis and Compilation Optimization: Overview and Prospects](https://mlsanity.com/paper/2407.00736/) (2024)
- [TensorRL-QAS: Reinforcement learning with tensor networks for improved quantum architecture search](https://mlsanity.com/paper/2505.09371/) (2025)
- [Optimizing the non-Clifford-count in unitary synthesis using Reinforcement Learning](https://mlsanity.com/paper/2509.21709/) (2025)
- [Quantum Compiling with Reinforcement Learning on a Superconducting Processor](https://mlsanity.com/paper/2406.12195/) (2024)
- [CNOT-Optimal Clifford Synthesis as SAT](https://mlsanity.com/paper/2504.00634/) (2025)
- [Software Architecture for Quantum Computing Systems -- A Systematic Review](https://mlsanity.com/paper/2202.05505/) (2022)
- [Improving thermal state preparation of Sachdev-Ye-Kitaev model with reinforcement learning on quantum hardware](https://mlsanity.com/paper/2501.11454/) (2025)
- [AlphaCNOT: Learning CNOT Minimization with Model-Based Planning](https://mlsanity.com/paper/2604.13812/) (2026)
- [Equivariant Reinforcement Learning for Clifford Quantum Circuit Synthesis](https://mlsanity.com/paper/2605.10910/) (2026)
- [Quantum Computing for Discrete Optimization: A Highlight of Three Technologies](https://mlsanity.com/paper/2409.01373/) (2024)
- [CNOT Minimal Circuit Synthesis: A Reinforcement Learning Approach](https://mlsanity.com/paper/2510.23304/) (2025)
- [Quantum Circuit Pre-Synthesis: Learning Local Edits to Reduce T-count](https://mlsanity.com/paper/2601.19738/) (2026)

## Abstract (arXiv)

Clifford circuits play a foundational role in quantum computing, particularly due to their importance in quantum error correction and fault-tolerant logical synthesis. While these circuits can be efficiently simulated and represented as symplectic matrices, standard synthesis methods-such as the Aaronson-Gottesman algorithm-often yield sub-optimal circuits with excessively high gate counts. In this work, we introduce AlphaClifford, a model-based Reinforcement Learning framework powered by Monte Carlo Tree Search, designed to efficiently synthesize Clifford circuits from the fundamental gate set composed of H, S, and CNOT. By modeling the state space through the algebraic properties of the symplectic group, AlphaClifford effectively explores this combinatorial space to minimize overall circuit cost. For unconstrained Clifford optimization, our approach achieves a consistent reduction in both total and two-qubit (CNOT) gate counts compared to state-of-the-art synthesis heuristics, despite operating with a strictly less expressive gate set. Furthermore, we demonstrate the broad applicability of our framework on two additional tasks: hardware-constrained Clifford transpilation, where we outperform existing RL-based compilers, and as a post-synthesis optimization component within a full Clifford+T logical synthesis pipeline. Our results underscore that model-based RL is highly effective at addressing the combinatorial complexities of quantum compilation, offering a scalable pathway to mitigate hardware constraints in both near-term and future fault-tolerant quantum devices.