# Expressive power of tensor-network factorizations for probabilistic modeling, with applications from hidden Markov models to quantum machine learning

arXiv: https://arxiv.org/abs/1907.03741 · HTML: https://mlsanity.com/paper/1907.03741/

## Key facts

- Cited by 55 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Jul 2019.

## Built on by (most cited first, 30 of 55)

- [Enhancing Generative Models via Quantum Correlations](https://mlsanity.com/paper/2101.08354/) (2021)
- [Subtractive Mixture Models via Squaring: Representation and Learning](https://mlsanity.com/paper/2310.00724/) (2023)
- [Tensor Networks Meet Neural Networks: A Survey and Future Perspectives](https://mlsanity.com/paper/2302.09019/) (2023)
- [Grokking phase transitions in learning local rules with gradient descent](https://mlsanity.com/paper/2210.15435/) (2022)
- [Potential and limitations of random Fourier features for dequantizing quantum machine learning](https://mlsanity.com/paper/2309.11647/) (2023)
- [What is the Relationship between Tensor Factorizations and Circuits (and How Can We Exploit it)?](https://mlsanity.com/paper/2409.07953/) (2024)
- [Tensor networks for unsupervised machine learning](https://mlsanity.com/paper/2106.12974/) (2021)
- [Sum of Squares Circuits](https://mlsanity.com/paper/2408.11778/) (2024)
- [Variational inference with a quantum computer](https://mlsanity.com/paper/2103.06720/) (2021)
- [Tensor networks for quantum machine learning](https://mlsanity.com/paper/2303.11735/) (2023)
- [Generative modeling with projected entangled-pair states](https://mlsanity.com/paper/2202.08177/) (2022)
- [On the Relationship Between Monotone and Squared Probabilistic Circuits](https://mlsanity.com/paper/2408.00876/) (2024)
- [Tensor networks and efficient descriptions of classical data](https://mlsanity.com/paper/2103.06872/) (2021)
- [Permutation Search of Tensor Network Structures via Local Sampling](https://mlsanity.com/paper/2206.06597/) (2022)
- [Alternating Local Enumeration (TnALE): Solving Tensor Network Structure Search with Fewer Evaluations](https://mlsanity.com/paper/2304.12875/) (2023)
- [SVDinsTN: A Tensor Network Paradigm for Efficient Structure Search from Regularized Modeling Perspective](https://mlsanity.com/paper/2305.14912/) (2023)
- [Generative Learning of Continuous Data by Tensor Networks](https://mlsanity.com/paper/2310.20498/) (2023)
- [SHAP Meets Tensor Networks: Provably Tractable Explanations with Parallelism](https://mlsanity.com/paper/2510.21599/) (2025)
- [Learnability of the output distributions of local quantum circuits](https://mlsanity.com/paper/2110.05517/) (2021)
- [Explainable Natural Language Processing with Matrix Product States](https://mlsanity.com/paper/2112.08628/) (2021)
- [A Matrix Product State Model for Simultaneous Classification and Generation](https://mlsanity.com/paper/2406.17441/) (2024)
- [Probabilistic Graphical Models and Tensor Networks: A Hybrid Framework](https://mlsanity.com/paper/2106.15666/) (2021)
- [Generative Modeling via Tree Tensor Network States](https://mlsanity.com/paper/2209.01341/) (2022)
- [Application of quantum-inspired generative models to small molecular datasets](https://mlsanity.com/paper/2304.10867/) (2023)
- [tnGPS: Discovering Unknown Tensor Network Structure Search Algorithms via Large Language Models (LLMs)](https://mlsanity.com/paper/2402.02456/) (2024)
- [How to Square Tensor Networks and Circuits Without Squaring Them](https://mlsanity.com/paper/2512.17090/) (2025)
- [Tensor network to learn the wavefunction of data](https://mlsanity.com/paper/2111.08014/) (2021)
- [F-Divergences and Cost Function Locality in Generative Modelling with Quantum Circuits](https://mlsanity.com/paper/2110.04253/) (2021)
- [Positive unlabeled learning with tensor networks](https://mlsanity.com/paper/2211.14085/) (2022)
- [Efficient Nonparametric Tensor Decomposition for Binary and Count Data](https://mlsanity.com/paper/2401.07711/) (2024)

## Abstract (arXiv)

Tensor-network techniques have enjoyed outstanding success in physics, and have recently attracted attention in machine learning, both as a tool for the formulation of new learning algorithms and for enhancing the mathematical understanding of existing methods. Inspired by these developments, and the natural correspondence between tensor networks and probabilistic graphical models, we provide a rigorous analysis of the expressive power of various tensor-network factorizations of discrete multivariate probability distributions. These factorizations include non-negative tensor-trains/MPS, which are in correspondence with hidden Markov models, and Born machines, which are naturally related to local quantum circuits. When used to model probability distributions, they exhibit tractable likelihoods and admit efficient learning algorithms. Interestingly, we prove that there exist probability distributions for which there are unbounded separations between the resource requirements of some of these tensor-network factorizations. Particularly surprising is the fact that using complex instead of real tensors can lead to an arbitrarily large reduction in the number of parameters of the network. Additionally, we introduce locally purified states (LPS), a new factorization inspired by techniques for the simulation of quantum systems, with provably better expressive power than all other representations considered. The ramifications of this result are explored through numerical experiments. Our findings imply that LPS should be considered over hidden Markov models, and furthermore provide guidelines for the design of local quantum circuits for probabilistic modeling.