# Deep tensor networks with matrix product operators

arXiv: https://arxiv.org/abs/2209.09098 · HTML: https://mlsanity.com/paper/2209.09098/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 48 references that are in the map.
- First posted on arXiv 16 Sep 2022.

## Builds on (most cited first, 30 of 34)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [The effect of data encoding on the expressive power of variational quantum machine learning models](https://mlsanity.com/paper/2008.08605/) (2020)
- [Tensorizing Neural Networks](https://mlsanity.com/paper/1509.06569/) (2015)
- [On the Expressive Power of Deep Learning: A Tensor Analysis](https://mlsanity.com/paper/1509.05009/) (2015)
- [Tree Tensor Networks for Generative Modeling](https://mlsanity.com/paper/1901.02217/) (2019)
- [Ultimate tensorization: compressing convolutional and FC layers alike](https://mlsanity.com/paper/1611.03214/) (2016)
- [Expressive power of tensor-network factorizations for probabilistic modeling, with applications from hidden Markov models to quantum machine learning](https://mlsanity.com/paper/1907.03741/) (2019)
- [Learning Relevant Features of Data with Multi-scale Tensor Networks](https://mlsanity.com/paper/1801.00315/) (2017)
- [Machine Learning by Unitary Tensor Network of Hierarchical Tree Structure](https://mlsanity.com/paper/1710.04833/) (2017)
- [Supervised Learning with Quantum-Inspired Tensor Networks](https://mlsanity.com/paper/1605.05775/) (2016)
- [Information Perspective to Probabilistic Modeling: Boltzmann Machines versus Born Machines](https://mlsanity.com/paper/1712.04144/) (2017)
- [Quantum-inspired Machine Learning on high-energy physics data](https://mlsanity.com/paper/2004.13747/) (2020)
- [Supervised Learning with Projected Entangled Pair States](https://mlsanity.com/paper/2009.09932/) (2020)
- [TensorNetwork for Machine Learning](https://mlsanity.com/paper/1906.06329/) (2019)
- [Equivalence of restricted Boltzmann machines and tensor network states](https://mlsanity.com/paper/1701.04831/) (2017)
- [Tensorized Embedding Layers for Efficient Model Compression](https://mlsanity.com/paper/1901.10787/) (2019)
- [Anomaly Detection with Tensor Networks](https://mlsanity.com/paper/2006.02516/) (2020)
- [Deep Learning and Quantum Entanglement: Fundamental Connections with Implications to Network Design](https://mlsanity.com/paper/1704.01552/) (2017)
- [Generative Tensor Network Classification Model for Supervised Machine Learning](https://mlsanity.com/paper/1903.10742/) (2019)
- [Tensor networks for unsupervised machine learning](https://mlsanity.com/paper/2106.12974/) (2021)
- [Modeling Sequences with Quantum States: A Look Under the Hood](https://mlsanity.com/paper/1910.07425/) (2019)
- [Entanglement and Tensor Networks for Supervised Image Classification](https://mlsanity.com/paper/2007.06082/) (2020)
- [Tensor networks and efficient descriptions of classical data](https://mlsanity.com/paper/2103.06872/) (2021)
- [Probabilistic Modeling with Matrix Product States](https://mlsanity.com/paper/1902.06888/) (2019)
- [Residual Matrix Product State for Machine Learning](https://mlsanity.com/paper/2012.11841/) (2020)
- [Tensor network language model](https://mlsanity.com/paper/1710.10248/) (2017)
- [Quantum Tensor Networks, Stochastic Processes, and Weighted Automata](https://mlsanity.com/paper/2010.10653/) (2020)
- [Quantum Tensor Network in Machine Learning: An Application to Tiny Object Classification](https://mlsanity.com/paper/2101.03154/) (2021)
- [Language Modeling with Reduced Densities](https://mlsanity.com/paper/2007.03834/) (2020)

## Built on by (most cited first, 2 of 2)

- [Grokking phase transitions in learning local rules with gradient descent](https://mlsanity.com/paper/2210.15435/) (2022)
- [Positive unlabeled learning with tensor networks](https://mlsanity.com/paper/2211.14085/) (2022)

## Abstract (arXiv)

We introduce deep tensor networks, which are exponentially wide neural networks based on the tensor network representation of the weight matrices. We evaluate the proposed method on the image classification (MNIST, FashionMNIST) and sequence prediction (cellular automata) tasks. In the image classification case, deep tensor networks improve our matrix product state baselines and achieve 0.49% error rate on MNIST and 8.3% error rate on FashionMNIST. In the sequence prediction case, we demonstrate an exponential improvement in the number of parameters compared to the one-layer tensor network methods. In both cases, we discuss the non-uniform and the uniform tensor network models and show that the latter generalizes well to different input sizes.