# Generalization and Overfitting in Matrix Product State Machine Learning Architectures

arXiv: https://arxiv.org/abs/2208.04372 · HTML: https://mlsanity.com/paper/2208.04372/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 28 references that are in the map.
- First posted on arXiv 8 Aug 2022.

## Builds on (most cited first, 7 of 7)

- [Deep Double Descent: Where Bigger Models and More Data Hurt](https://mlsanity.com/paper/1912.02292/) (2019)
- [More Data Can Hurt for Linear Regression: Sample-wise Double Descent](https://mlsanity.com/paper/1912.07242/) (2019)
- [From probabilistic graphical models to generalized tensor networks for supervised learning](https://mlsanity.com/paper/1806.05964/) (2018)
- [Tensor networks for unsupervised machine learning](https://mlsanity.com/paper/2106.12974/) (2021)
- [Entanglement and Tensor Networks for Supervised Image Classification](https://mlsanity.com/paper/2007.06082/) (2020)
- [Tensor networks and efficient descriptions of classical data](https://mlsanity.com/paper/2103.06872/) (2021)
- [Generative modeling via tensor train sketching](https://mlsanity.com/paper/2202.11788/) (2022)

## Built on by (most cited first, 3 of 3)

- [Machine learning with tree tensor networks, CP rank constraints, and tensor dropout](https://mlsanity.com/paper/2305.19440/) (2023)
- [No-Free-Lunch Theories for Tensor-Network Machine Learning Models](https://mlsanity.com/paper/2412.05674/) (2024)
- [Regularized second-order optimization of tensor-network Born machines](https://mlsanity.com/paper/2501.18691/) (2025)

## Abstract (arXiv)

While overfitting and, more generally, double descent are ubiquitous in machine learning, increasing the number of parameters of the most widely used tensor network, the matrix product state (MPS), has generally lead to monotonic improvement of test performance in previous studies. To better understand the generalization properties of architectures parameterized by MPS, we construct artificial data which can be exactly modeled by an MPS and train the models with different number of parameters. We observe model overfitting for one-dimensional data, but also find that for more complex data overfitting is less significant, while with MNIST image data we do not find any signatures of overfitting. We speculate that generalization properties of MPS depend on the properties of data: with one-dimensional data (for which the MPS ansatz is the most suitable) MPS is prone to overfitting, while with more complex data which cannot be fit by MPS exactly, overfitting may be much less significant.