# Machine learning with tree tensor networks, CP rank constraints, and tensor dropout

arXiv: https://arxiv.org/abs/2305.19440 · HTML: https://mlsanity.com/paper/2305.19440/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 89 references that are in the map.
- First posted on arXiv 30 May 2023.

## Builds on (most cited first, 29 of 29)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [On the Convergence of Adam and Beyond](https://mlsanity.com/paper/1904.09237/) (2019)
- [Identifying and attacking the saddle point problem in high-dimensional non-convex optimization](https://mlsanity.com/paper/1406.2572/) (2014)
- [On the Expressive Power of Deep Learning: A Tensor Analysis](https://mlsanity.com/paper/1509.05009/) (2015)
- [Unsupervised Generative Modeling Using Matrix Product States](https://mlsanity.com/paper/1709.01662/) (2017)
- [Failures of Gradient-Based Deep Learning](https://mlsanity.com/paper/1703.07950/) (2017)
- [Towards Quantum Machine Learning with Tensor Networks](https://mlsanity.com/paper/1803.11537/) (2018)
- [Tree Tensor Networks for Generative Modeling](https://mlsanity.com/paper/1901.02217/) (2019)
- [Learning Relevant Features of Data with Multi-scale Tensor Networks](https://mlsanity.com/paper/1801.00315/) (2017)
- [Machine Learning by Unitary Tensor Network of Hierarchical Tree Structure](https://mlsanity.com/paper/1710.04833/) (2017)
- [From probabilistic graphical models to generalized tensor networks for supervised learning](https://mlsanity.com/paper/1806.05964/) (2018)
- [Supervised Learning with Projected Entangled Pair States](https://mlsanity.com/paper/2009.09932/) (2020)
- [TensorNetwork for Machine Learning](https://mlsanity.com/paper/1906.06329/) (2019)
- [Hybrid quantum-classical classifier based on tensor network and variational quantum circuit](https://mlsanity.com/paper/2011.14651/) (2020)
- [Generative Tensor Network Classification Model for Supervised Machine Learning](https://mlsanity.com/paper/1903.10742/) (2019)
- [Tensor networks for unsupervised machine learning](https://mlsanity.com/paper/2106.12974/) (2021)
- [Tensor Networks for Medical Image Classification](https://mlsanity.com/paper/2004.10076/) (2020)
- [Entanglement and Tensor Networks for Supervised Image Classification](https://mlsanity.com/paper/2007.06082/) (2020)
- [Generative modeling with projected entangled-pair states](https://mlsanity.com/paper/2202.08177/) (2022)
- [Entanglement-guided architectures of machine learning by quantum tensor network](https://mlsanity.com/paper/1803.09111/) (2018)
- [Quantum Tensor Network in Machine Learning: An Application to Tiny Object Classification](https://mlsanity.com/paper/2101.03154/) (2021)
- [Quantum-inspired event reconstruction with Tensor Networks: Matrix Product States](https://mlsanity.com/paper/2106.08334/) (2021)
- [Unsupervised Recognition of Informative Features via Tensor Network Machine Learning and Quantum Entanglement Variations](https://mlsanity.com/paper/2207.06031/) (2022)
- [Tensor network to learn the wavefunction of data](https://mlsanity.com/paper/2111.08014/) (2021)
- [Interaction Decompositions for Tensor Network Regression](https://mlsanity.com/paper/2208.06029/) (2022)
- [Generalization and Overfitting in Matrix Product State Machine Learning Architectures](https://mlsanity.com/paper/2208.04372/) (2022)
- [Density peak clustering using tensor network](https://mlsanity.com/paper/2302.00192/) (2023)

## Built on by (most cited first, 15 of 15)

- [SHAP Meets Tensor Networks: Provably Tractable Explanations with Parallelism](https://mlsanity.com/paper/2510.21599/) (2025)
- [Ultra-low latency quantum-inspired machine learning predictors implemented on FPGA](https://mlsanity.com/paper/2409.16075/) (2024)
- [Tensorization is a powerful but underexplored tool for compression and interpretability of neural networks](https://mlsanity.com/paper/2505.20132/) (2025)
- [Riemannian Optimization on Tree Tensor Networks with Application in Machine Learning](https://mlsanity.com/paper/2507.21726/) (2025)
- [Universal scaling laws in quantum-probabilistic machine learning by tensor network towards interpreting representation and generalization powers](https://mlsanity.com/paper/2410.09703/) (2024)
- [Web Service QoS Prediction via Extended Canonical Polyadic-based Tensor Network](https://mlsanity.com/paper/2408.16278/) (2024)
- [Trainable Quantum Neural Network for Multiclass Image Classification with the Power of Pre-trained Tree Tensor Networks](https://mlsanity.com/paper/2504.14995/) (2025)
- [A PID-Controlled Tensor Wheel Decomposition Model for Dynamic Link Prediction](https://mlsanity.com/paper/2505.14211/) (2025)
- [Fast and Accurate Power Load Data Completion via Regularization-optimized Low-Rank Factorization](https://mlsanity.com/paper/2505.19133/) (2025)
- [Accelerating Large-Scale Regularized High-Order Tensor Recovery](https://mlsanity.com/paper/2506.09594/) (2025)
- [Tensor-Augmented Convolutional Neural Networks: Enhancing Expressivity with Generic Tensor Kernels](https://mlsanity.com/paper/2604.08072/) (2026)
- [Quantum-inspired tensor networks in machine learning models](https://mlsanity.com/paper/2604.14287/) (2026)
- [Beyond Backpropagation: Monte Carlo Method Can Train Deep Neural Networks](https://mlsanity.com/paper/2607.08406/) (2026)
- [Stochastic Optimization of Tree Tensor Networks](https://mlsanity.com/paper/2609.00870/) (2026)
- [Gaussian Universality and Its Breakdown in Tensor-Network Machine Learning](https://mlsanity.com/paper/2610.06080/) (2026)

## Abstract (arXiv)

Tensor networks developed in the context of condensed matter physics try to approximate order-N tensors with a reduced number of degrees of freedom that is only polynomial in N and arranged as a network of partially contracted smaller tensors. As we have recently demonstrated in the context of quantum many-body physics, computation costs can be further substantially reduced by imposing constraints on the canonical polyadic (CP) rank of the tensors in such networks [arXiv:2205.15296]. Here, we demonstrate how tree tensor networks (TTN) with CP rank constraints and tensor dropout can be used in machine learning. The approach is found to outperform other tensor-network-based methods in Fashion-MNIST image classification. A low-rank TTN classifier with branching ratio b=4 reaches a test set accuracy of 90.3% with low computation costs. Consisting of mostly linear elements, tensor network classifiers avoid the vanishing gradient problem of deep neural networks. The CP rank constraints have additional advantages: The number of parameters can be decreased and tuned more freely to control overfitting, improve generalization properties, and reduce computation costs. They allow us to employ trees with large branching ratios, substantially improving the representation power.