# Efficient Finite Initialization with Partial Norms for Tensorized Neural Networks and Tensor Networks Algorithms

arXiv: https://arxiv.org/abs/2309.06577 · HTML: https://mlsanity.com/paper/2309.06577/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 7 of its 26 references that are in the map.
- First posted on arXiv 11 Sep 2023.

## Builds on (most cited first, 7 of 7)

- [Compressing deep neural networks by matrix product operators](https://mlsanity.com/paper/1904.06194/) (2019)
- [Anomaly Detection with Tensor Networks](https://mlsanity.com/paper/2006.02516/) (2020)
- [Exploiting Low-Rank Tensor-Train Deep Neural Networks Based on Riemannian Gradient Descent With Illustrations of Speech Processing](https://mlsanity.com/paper/2203.06031/) (2022)
- [Quantum-Inspired Tensor Neural Networks for Partial Differential Equations](https://mlsanity.com/paper/2208.02235/) (2022)
- [Compressing Neural Networks Using Tensor Networks with Exponentially Fewer Variational Parameters](https://mlsanity.com/paper/2305.06058/) (2023)
- [A quantum-inspired tensor network method for constrained combinatorial optimization problems](https://mlsanity.com/paper/2203.15246/) (2022)
- [Improvements to Gradient Descent Methods for Quantum Tensor Network Machine Learning](https://mlsanity.com/paper/2203.03366/) (2022)

## Abstract (arXiv)

We present two algorithms to initialize layers of tensorized neural networks and general tensor network algorithms using partial computations of their Frobenius norms and positive lineal entrywise sums, depending on the type of tensor network involved. The core of this method is the use of the norm of subnetworks of the tensor network in an iterative way, so that we normalize by the finite values of the norms that led to the divergence or zero norm. In addition, the method benefits from the reuse of intermediate calculations. We have also applied it to the Matrix Product State/Tensor Train (MPS/TT) and Matrix Product Operator/Tensor Train Matrix (MPO/TT-M) layers and have seen its scaling versus the number of nodes, bond dimension, and physical dimension. All code is publicly available.