# Compilation of product-formula Hamiltonian simulation via reinforcement learning

arXiv: https://arxiv.org/abs/2311.04285 · HTML: https://mlsanity.com/paper/2311.04285/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 14 of its 61 references that are in the map.
- First posted on arXiv 7 Nov 2023.

## Builds on (most cited first, 14 of 14)

- [Reinforcement learning for optimization of variational quantum circuit architectures](https://mlsanity.com/paper/2103.16089/) (2021)
- [Optimizing Quantum Error Correction Codes with Reinforcement Learning](https://mlsanity.com/paper/1812.08451/) (2018)
- [Topological Quantum Compiling with Reinforcement Learning](https://mlsanity.com/paper/2004.04743/) (2020)
- [Graph-theoretic Simplification of Quantum Circuits with the ZX-calculus](https://mlsanity.com/paper/1902.03178/) (2019)
- [Automated quantum programming via reinforcement learning for combinatorial optimization](https://mlsanity.com/paper/1908.08054/) (2019)
- [Reinforcement-Learning-Based Variational Quantum Circuits Optimization for Combinatorial Problems](https://mlsanity.com/paper/1911.04574/) (2019)
- [Quantum Logic Gate Synthesis as a Markov Decision Process](https://mlsanity.com/paper/1912.12002/) (2019)
- [Compiling quantum circuits to realistic hardware architectures using temporal planners](https://mlsanity.com/paper/1705.08927/) (2017)
- [Comparing and Integrating Constraint Programming and Temporal Planning for Quantum Circuit Compilation](https://mlsanity.com/paper/1803.06775/) (2018)
- [Reinforcement Learning Enhanced Quantum-inspired Algorithm for Combinatorial Optimization](https://mlsanity.com/paper/2002.04676/) (2020)
- [A randomized quantum algorithm for statistical phase estimation](https://mlsanity.com/paper/2110.12071/) (2021)
- [Partial Compilation of Variational Algorithms for Noisy Intermediate-Scale Quantum Machines](https://mlsanity.com/paper/1909.07522/) (2019)
- [Efficient and Effective Quantum Compiling for Entanglement-based Machine Learning on IBM Q Devices](https://mlsanity.com/paper/1801.02363/) (2018)
- [Synthesizing efficient circuits for Hamiltonian simulation](https://mlsanity.com/paper/2209.03478/) (2022)

## Abstract (arXiv)

Hamiltonian simulation is believed to be one of the first tasks where quantum computers can yield a quantum advantage. One of the most popular methods of Hamiltonian simulation is Trotterization, which makes use of the approximation eisumjAj~ prodjeiAj and higher-order corrections thereto. However, this leaves open the question of the order of operations (i.e. the order of the product over j, which is known to affect the quality of approximation). In some cases this order is fixed by the desire to minimise the error of approximation; when it is not the case, we propose that the order can be chosen to optimize compilation to a native quantum architecture. This presents a new compilation problem -- order-agnostic quantum circuit compilation -- which we prove is NP-hard in the worst case. In lieu of an easily-computable exact solution, we turn to methods of heuristic optimization of compilation. We focus on reinforcement learning due to the sequential nature of the compilation task, comparing it to simulated annealing and Monte Carlo tree search. While two of the methods outperform a naive heuristic, reinforcement learning clearly outperforms all others, with a gain of around 12% with respect to the second-best method and of around 50% compared to the naive heuristic in terms of the gate count. We further test the ability of RL to generalize across instances of the compilation problem, and find that a single learner is able to solve entire problem families. This demonstrates the ability of machine learning techniques to provide assistance in an order-agnostic quantum compilation task.