# A Tensor Network Implementation of Multi Agent Reinforcement Learning

arXiv: https://arxiv.org/abs/2401.03896 · HTML: https://mlsanity.com/paper/2401.03896/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 5 of its 49 references that are in the map.
- First posted on arXiv 8 Jan 2024.

## Builds on (most cited first, 5 of 5)

- [Expressive power of tensor-network factorizations for probabilistic modeling, with applications from hidden Markov models to quantum machine learning](https://mlsanity.com/paper/1907.03741/) (2019)
- [TensorNetwork for Machine Learning](https://mlsanity.com/paper/1906.06329/) (2019)
- [TensorNetwork: A Library for Physics and Machine Learning](https://mlsanity.com/paper/1905.01330/) (2019)
- [Compact Neural Networks based on the Multiscale Entanglement Renormalization Ansatz](https://mlsanity.com/paper/1711.03357/) (2017)
- [A Tensor Network Approach to Finite Markov Decision Processes](https://mlsanity.com/paper/2002.05185/) (2020)

## Abstract (arXiv)

Recently it has been shown that tensor networks (TNs) have the ability to represent the expected return of a single-agent finite Markov decision process (FMDP). The TN represents a distribution model, where all possible trajectories are considered. When extending these ideas to a multi-agent setting, distribution models suffer from the curse of dimensionality: the exponential relation between the number of possible trajectories and the number of agents. The key advantage of using TNs in this setting is that there exists a large number of established optimisation and decomposition techniques that are specific to TNs, that one can apply to ensure the most efficient representation is found. In this report, these methods are used to form a TN that represents the expected return of a multi-agent reinforcement learning (MARL) task. This model is then applied to a 2 agent random walker example, where it was shown that the policy is correctly optimised using a DMRG technique. Finally, I demonstrate the use of an exact decomposition technique, reducing the number of elements in the tensors by 97.5%, without experiencing any loss of information.