# Optimizing ZX-Diagrams with Deep Reinforcement Learning

arXiv: https://arxiv.org/abs/2311.18588 · HTML: https://mlsanity.com/paper/2311.18588/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 45 references that are in the map.
- First posted on arXiv 30 Nov 2023.

## Builds on (most cited first, 15 of 15)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Neural Message Passing for Quantum Chemistry](https://mlsanity.com/paper/1704.01212/) (2017)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Graph Neural Networks: A Review of Methods and Applications](https://mlsanity.com/paper/1812.08434/) (2018)
- [Learning Dexterous In-Hand Manipulation](https://mlsanity.com/paper/1808.00177/) (2018)
- [Graph Convolutional Policy Network for Goal-Directed Molecular Graph Generation](https://mlsanity.com/paper/1806.02473/) (2018)
- [GraphAF: a Flow-based Autoregressive Model for Molecular Graph Generation](https://mlsanity.com/paper/2001.09382/) (2020)
- [Reinforcement Learning Decoders for Fault-Tolerant Quantum Computation](https://mlsanity.com/paper/1810.07207/) (2018)
- [TF-GNN: Graph Neural Networks in TensorFlow](https://mlsanity.com/paper/2207.03522/) (2022)
- [Graph-theoretic Simplification of Quantum Circuits with the ZX-calculus](https://mlsanity.com/paper/1902.03178/) (2019)
- [Quarl: A Learning-Based Quantum Circuit Optimizer](https://mlsanity.com/paper/2307.10120/) (2023)
- [Reducing 2-QuBit Gate Count for ZX-Calculus based Quantum Circuit Optimization](https://mlsanity.com/paper/2311.08881/) (2023)
- [Verifying the Smallest Interesting Colour Code with Quantomatic](https://mlsanity.com/paper/1706.02717/) (2017)
- [Relating Measurement Patterns to Circuits via Pauli Flow](https://mlsanity.com/paper/2109.05654/) (2021)

## Built on by (most cited first, 6 of 6)

- [Meta-Designing Quantum Experiments with Language Models](https://mlsanity.com/paper/2406.02470/) (2024)
- [Optimizing Quantum Circuits via ZX Diagrams using Reinforcement Learning and Graph Neural Networks](https://mlsanity.com/paper/2504.03429/) (2025)
- [Equivariant Reinforcement Learning for Clifford Quantum Circuit Synthesis](https://mlsanity.com/paper/2605.10910/) (2026)
- [Automatic re-calibration of quantum devices by reinforcement learning](https://mlsanity.com/paper/2404.10726/) (2024)
- [Tackling Decision Processes with Non-Cumulative Objectives using Reinforcement Learning](https://mlsanity.com/paper/2405.13609/) (2024)
- [Reinforcement Learning for Adaptive Composition of Quantum Circuit Optimisation Passes](https://mlsanity.com/paper/2601.21629/) (2026)

## Abstract (arXiv)

ZX-diagrams are a powerful graphical language for the description of quantum processes with applications in fundamental quantum mechanics, quantum circuit optimization, tensor network simulation, and many more. The utility of ZX-diagrams relies on a set of local transformation rules that can be applied to them without changing the underlying quantum process they describe. These rules can be exploited to optimize the structure of ZX-diagrams for a range of applications. However, finding an optimal sequence of transformation rules is generally an open problem. In this work, we bring together ZX-diagrams with reinforcement learning, a machine learning technique designed to discover an optimal sequence of actions in a decision-making problem and show that a trained reinforcement learning agent can significantly outperform other optimization techniques like a greedy strategy, simulated annealing, and state-of-the-art hand-crafted algorithms. The use of graph neural networks to encode the policy of the agent enables generalization to diagrams much bigger than seen during the training phase.