# Equivariant Reinforcement Learning for Clifford Quantum Circuit Synthesis

arXiv: https://arxiv.org/abs/2605.10910 · HTML: https://mlsanity.com/paper/2605.10910/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Quantum reinforcement learning, one of 15 directions in Deep reinforcement learning methods.
- Builds on 14 of its 44 references that are in the map.
- First posted on arXiv 11 May 2026.

## Placed in

- Deep reinforcement learning methods / [Quantum reinforcement learning](https://mlsanity.com/deeprl/quantum-reinforcement-learning/)

## Builds on (most cited first, 14 of 14)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Self-Attention with Relative Position Representations](https://mlsanity.com/paper/1803.02155/) (2018)
- [Curriculum Learning for Reinforcement Learning Domains: A Framework and Survey](https://mlsanity.com/paper/2003.04960/) (2020)
- [Reverse Curriculum Generation for Reinforcement Learning](https://mlsanity.com/paper/1707.05300/) (2017)
- [MDP Homomorphic Networks: Group Symmetries in Reinforcement Learning](https://mlsanity.com/paper/2006.16908/) (2020)
- [Quantum Circuit Optimization with AlphaTensor](https://mlsanity.com/paper/2402.14396/) (2024)
- [PufferLib: Making Reinforcement Learning Libraries and Environments Play Nice](https://mlsanity.com/paper/2406.12905/) (2024)
- [Graph-theoretic Simplification of Quantum Circuits with the ZX-calculus](https://mlsanity.com/paper/1902.03178/) (2019)
- [Optimizing ZX-Diagrams with Deep Reinforcement Learning](https://mlsanity.com/paper/2311.18588/) (2023)
- [CNOT-Optimal Clifford Synthesis as SAT](https://mlsanity.com/paper/2504.00634/) (2025)
- [Relating Measurement Patterns to Circuits via Pauli Flow](https://mlsanity.com/paper/2109.05654/) (2021)
- [Optimizing Quantum Circuits via ZX Diagrams using Reinforcement Learning and Graph Neural Networks](https://mlsanity.com/paper/2504.03429/) (2025)
- [AlphaCNOT: Learning CNOT Minimization with Model-Based Planning](https://mlsanity.com/paper/2604.13812/) (2026)

## Built on by (most cited first, 2 of 2)

- [AlphaClifford: Efficient Clifford Synthesis and Transpilation with Model-based RL](https://mlsanity.com/paper/2608.18946/) (2026)
- [Bridge of Psi's: Quantum Circuit Optimization with Schr"odinger Bridges](https://mlsanity.com/paper/2609.25947/) (2026)

## Abstract (arXiv)

We consider the problem of synthesizing Clifford quantum circuits for devices with all-to-all qubit connectivity. We approach this task as a reinforcement learning problem in which an agent learns to discover a sequence of elementary Clifford gates that reduces a given symplectic matrix representation of a Clifford circuit to the identity. This formulation permits a simple learning curriculum based on random walks from the identity. We introduce a novel neural network architecture that is equivariant to qubit relabelings of the symplectic matrix representation, and which is size-agnostic, allowing a single learned policy to be applied across different qubit counts without circuit splicing or network reparameterization. On six-qubit Clifford circuits, the largest regime for which optimal references are available, our agent finds circuits within one two-qubit gate of optimality in milliseconds per instance, and finds optimal circuits in 99.2% of instances within seconds per instance. After continued training on ten-qubit instances, the agent scales to unseen Clifford tableaus with up to thirty qubits, including targets generated from circuits with over a thousand Clifford gates, where it achieves lower average two-qubit gate counts than Qiskit's Aaronson-Gottesman and greedy Clifford synthesizers.