# Universal scaling laws in quantum-probabilistic machine learning by tensor network towards interpreting representation and generalization powers

arXiv: https://arxiv.org/abs/2410.09703 · HTML: https://mlsanity.com/paper/2410.09703/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 41 references that are in the map.
- First posted on arXiv 13 Oct 2024.

## Builds on (most cited first, 20 of 20)

- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Training Compute-Optimal Large Language Models](https://mlsanity.com/paper/2203.15556/) (2022)
- [Scaling Laws for Reward Model Overoptimization](https://mlsanity.com/paper/2210.10760/) (2022)
- [Scaling Vision Transformers](https://mlsanity.com/paper/2106.04560/) (2021)
- [Beyond neural scaling laws: beating power law scaling via data pruning](https://mlsanity.com/paper/2206.14486/) (2022)
- [Scaling Laws for Autoregressive Generative Modeling](https://mlsanity.com/paper/2010.14701/) (2020)
- [Explaining Explanations: An Overview of Interpretability of Machine Learning](https://mlsanity.com/paper/1806.00069/) (2018)
- [Deep Learning for Classical Japanese Literature](https://mlsanity.com/paper/1812.01718/) (2018)
- [Scaling Data-Constrained Language Models](https://mlsanity.com/paper/2305.16264/) (2023)
- [Visual Interpretability for Deep Learning: a Survey](https://mlsanity.com/paper/1802.00614/) (2018)
- [Tree Tensor Networks for Generative Modeling](https://mlsanity.com/paper/1901.02217/) (2019)
- [Machine Learning by Unitary Tensor Network of Hierarchical Tree Structure](https://mlsanity.com/paper/1710.04833/) (2017)
- [Generative Tensor Network Classification Model for Supervised Machine Learning](https://mlsanity.com/paper/1903.10742/) (2019)
- [Tensor networks for unsupervised machine learning](https://mlsanity.com/paper/2106.12974/) (2021)
- [Tensor networks for interpretable and efficient quantum-inspired machine learning](https://mlsanity.com/paper/2311.11258/) (2023)
- [Machine learning with tree tensor networks, CP rank constraints, and tensor dropout](https://mlsanity.com/paper/2305.19440/) (2023)
- [Residual Matrix Product State for Machine Learning](https://mlsanity.com/paper/2012.11841/) (2020)
- [Privacy-preserving machine learning with tensor networks](https://mlsanity.com/paper/2202.12319/) (2022)
- [Unsupervised Recognition of Informative Features via Tensor Network Machine Learning and Quantum Entanglement Variations](https://mlsanity.com/paper/2207.06031/) (2022)

## Built on by (most cited first, 1 of 1)

- [Interpretable Machine Learning in Physics: A Review](https://mlsanity.com/paper/2503.23616/) (2025)

## Abstract (arXiv)

Interpreting the representation and generalization powers has been a long-standing issue in the field of machine learning (ML) and artificial intelligence. This work contributes to uncovering the emergence of universal scaling laws in quantum-probabilistic ML. We take the generative tensor network (GTN) in the form of a matrix product state as an example and show that with an untrained GTN (such as a random TN state), the negative logarithmic likelihood (NLL) L generally increases linearly with the number of features M, i.e., L simeq k M + const. This is a consequence of the so-called ``catastrophe of orthogonality,'' which states that quantum many-body states tend to become exponentially orthogonal to each other as M increases. We reveal that while gaining information through training, the linear scaling law is suppressed by a negative quadratic correction, leading to L simeq β M - α M2 + const. The scaling coefficients exhibit logarithmic relationships with the number of training samples and the number of quantum channels chi. The emergence of the quadratic correction term in NLL for the testing (training) set can be regarded as evidence of the generalization (representation) power of GTN. Over-parameterization can be identified by the deviation in the values of α between training and testing sets while increasing chi. We further investigate how orthogonality in the quantum feature map relates to the satisfaction of quantum probabilistic interpretation, as well as to the representation and generalization powers of GTN. The unveiling of universal scaling laws in quantum-probabilistic ML would be a valuable step toward establishing a white-box ML scheme interpreted within the quantum probabilistic framework.