# Identifying optimal cycles in quantum thermal machines with reinforcement-learning

arXiv: https://arxiv.org/abs/2108.13525 · HTML: https://mlsanity.com/paper/2108.13525/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 113 references that are in the map.
- First posted on arXiv 30 Aug 2021.

## Builds on (most cited first, 8 of 8)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Soft Actor-Critic Algorithms and Applications](https://mlsanity.com/paper/1812.05905/) (2018)
- [Soft Actor-Critic for Discrete Action Settings](https://mlsanity.com/paper/1910.07207/) (2019)
- [Learning to Walk via Deep Reinforcement Learning](https://mlsanity.com/paper/1812.11103/) (2018)
- [Universal Quantum Control through Deep Reinforcement Learning](https://mlsanity.com/paper/1803.01857/) (2018)
- [Discrete and Continuous Action Representation for Practical RL in Video Games](https://mlsanity.com/paper/1912.11077/) (2019)
- [Reinforcement Learning Decoders for Fault-Tolerant Quantum Computation](https://mlsanity.com/paper/1810.07207/) (2018)

## Built on by (most cited first, 6 of 6)

- [Equivariant quantum circuits for learning on weighted graphs](https://mlsanity.com/paper/2205.06109/) (2022)
- [Model-free optimization of power/efficiency tradeoffs in quantum thermal machines using reinforcement learning](https://mlsanity.com/paper/2204.04785/) (2022)
- [Shuffle-QUDIO: accelerate distributed VQE with trainability enhancement and measurement reduction](https://mlsanity.com/paper/2209.12454/) (2022)
- [Artificially intelligent Maxwell's demon for optimal control of open quantum systems](https://mlsanity.com/paper/2408.15328/) (2024)
- [Learning coherences from nonequilibrium fluctuations in a quantum heat engine](https://mlsanity.com/paper/2302.13717/) (2023)
- [Optimization of geological carbon storage operations with multimodal latent dynamic model and deep reinforcement learning](https://mlsanity.com/paper/2406.04575/) (2024)

## Abstract (arXiv)

The optimal control of open quantum systems is a challenging task but has a key role in improving existing quantum information processing technologies. We introduce a general framework based on Reinforcement Learning to discover optimal thermodynamic cycles that maximize the power of out-of-equilibrium quantum heat engines and refrigerators. We apply our method, based on the soft actor-critic algorithm, to three systems: a benchmark two-level system heat engine, where we find the optimal known cycle; an experimentally realistic refrigerator based on a superconducting qubit that generates coherence, where we find a non-intuitive control sequence that outperform previous cycles proposed in literature; a heat engine based on a quantum harmonic oscillator, where we find a cycle with an elaborate structure that outperforms the optimized Otto cycle. We then evaluate the corresponding efficiency at maximum power.