# Improvements to Gradient Descent Methods for Quantum Tensor Network Machine Learning

arXiv: https://arxiv.org/abs/2203.03366 · HTML: https://mlsanity.com/paper/2203.03366/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 22 references that are in the map.
- First posted on arXiv 3 Mar 2022.

## Builds on (most cited first, 12 of 12)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [DARTS: Differentiable Architecture Search](https://mlsanity.com/paper/1806.09055/) (2018)
- [Pruning and Quantization for Deep Neural Network Acceleration: A Survey](https://mlsanity.com/paper/2101.09671/) (2021)
- [Tensorizing Neural Networks](https://mlsanity.com/paper/1509.06569/) (2015)
- [Tree Tensor Networks for Generative Modeling](https://mlsanity.com/paper/1901.02217/) (2019)
- [Supervised Learning with Projected Entangled Pair States](https://mlsanity.com/paper/2009.09932/) (2020)
- [Anomaly Detection with Tensor Networks](https://mlsanity.com/paper/2006.02516/) (2020)
- [A Multi-Scale Tensor Network Architecture for Classification and Regression](https://mlsanity.com/paper/2001.08286/) (2020)
- [Tangent-Space Gradient Optimization of Tensor Network for Machine Learning](https://mlsanity.com/paper/2001.04029/) (2020)

## Built on by (most cited first, 7 of 7)

- [Tensor networks for quantum machine learning](https://mlsanity.com/paper/2303.11735/) (2023)
- [MetaTT: A Global Tensor-Train Adapter for Parameter-Efficient Fine-Tuning](https://mlsanity.com/paper/2506.09105/) (2025)
- [Tensor Network-Constrained Kernel Machines as Gaussian Processes](https://mlsanity.com/paper/2403.19500/) (2024)
- [Tensorization of neural networks for improved privacy and interpretability](https://mlsanity.com/paper/2501.06300/) (2025)
- [Tensor Methods for Language Models: From Token Representation to Training, Adaptation, Inference, Compression, and Interpretability](https://mlsanity.com/paper/2608.30505/) (2026)
- [Efficient Finite Initialization with Partial Norms for Tensorized Neural Networks and Tensor Networks Algorithms](https://mlsanity.com/paper/2309.06577/) (2023)
- [Entanglement is Half the Story: Post-Selection vs. Partial Traces](https://mlsanity.com/paper/2605.02385/) (2026)

## Abstract (arXiv)

Tensor networks have demonstrated significant value for machine learning in a myriad of different applications. However, optimizing tensor networks using standard gradient descent has proven to be difficult in practice. Tensor networks suffer from initialization problems resulting in exploding or vanishing gradients and require extensive hyperparameter tuning. Efforts to overcome these problems usually depend on specific network architectures, or ad hoc prescriptions. In this paper we address the problems of initialization and hyperparameter tuning, making it possible to train tensor networks using established machine learning techniques. We introduce a `copy node' method that successfully initializes arbitrary tensor networks, in addition to a gradient based regularization technique for bond dimensions. We present numerical results that show that the combination of techniques presented here produces quantum inspired tensor network models with far fewer parameters, while improving generalization performance.