# Combining Reinforcement Learning and Tensor Networks, with an Application to Dynamical Large Deviations

arXiv: https://arxiv.org/abs/2209.14089 · HTML: https://mlsanity.com/paper/2209.14089/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 41 references that are in the map.
- First posted on arXiv 28 Sep 2022.

## Builds on (most cited first, 5 of 5)

- [Learning Relevant Features of Data with Multi-scale Tensor Networks](https://mlsanity.com/paper/1801.00315/) (2017)
- [Adaptive importance sampling for control and inference](https://mlsanity.com/paper/1505.01874/) (2015)
- [A reinforcement learning approach to rare trajectory sampling](https://mlsanity.com/paper/2005.12890/) (2020)
- [Reinforcement learning of rare diffusive dynamics](https://mlsanity.com/paper/2105.04321/) (2021)
- [Evolutionary reinforcement learning of dynamical large deviations](https://mlsanity.com/paper/1909.00835/) (2019)

## Built on by (most cited first, 4 of 4)

- [Rare Event Analysis of Large Language Models](https://mlsanity.com/paper/2602.06791/) (2026)
- [Tensor tree learns hidden relational structures in data to construct generative models](https://mlsanity.com/paper/2408.10669/) (2024)
- [Understanding the Nature of Depth-1 Equivariant Quantum Circuit](https://mlsanity.com/paper/2511.10756/) (2025)
- [Thermodynamics of Reinforcement Learning Curricula](https://mlsanity.com/paper/2603.12324/) (2026)

## Abstract (arXiv)

We present a framework to integrate tensor network (TN) methods with reinforcement learning (RL) for solving dynamical optimisation tasks. We consider the RL actor-critic method, a model-free approach for solving RL problems, and introduce TNs as the approximators for its policy and value functions. Our "actor-critic with tensor networks" (ACTeN) method is especially well suited to problems with large and factorisable state and action spaces. As an illustration of the applicability of ACTeN we solve the exponentially hard task of sampling rare trajectories in two paradigmatic stochastic models, the East model of glasses and the asymmetric simple exclusion process (ASEP), the latter being particularly challenging to other methods due to the absence of detailed balance. With substantial potential for further integration with the vast array of existing RL methods, the approach introduced here is promising both for applications in physics and to multi-agent RL problems more generally.