# Residual Tensor Train: A Quantum-inspired Approach for Learning Multiple Multilinear Correlations

arXiv: https://arxiv.org/abs/2108.08659 · HTML: https://mlsanity.com/paper/2108.08659/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 63 references that are in the map.
- First posted on arXiv 19 Aug 2021.

## Builds on (most cited first, 19 of 19)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Identity Mappings in Deep Residual Networks](https://mlsanity.com/paper/1603.05027/) (2016)
- [Exact solutions to the nonlinear dynamics of learning in deep linear neural networks](https://mlsanity.com/paper/1312.6120/) (2013)
- [Bilinear CNNs for Fine-grained Visual Recognition](https://mlsanity.com/paper/1504.07889/) (2015)
- [Deep Information Propagation](https://mlsanity.com/paper/1611.01232/) (2016)
- [Compact Bilinear Pooling](https://mlsanity.com/paper/1511.06062/) (2015)
- [Looking for the Devil in the Details: Learning Trilinear Attention Sampling Network for Fine-grained Image Recognition](https://mlsanity.com/paper/1903.06150/) (2019)
- [Unsupervised Generative Modeling Using Matrix Product States](https://mlsanity.com/paper/1709.01662/) (2017)
- [Towards Quantum Machine Learning with Tensor Networks](https://mlsanity.com/paper/1803.11537/) (2018)
- [Deep Multi-task Representation Learning: A Tensor Factorisation Approach](https://mlsanity.com/paper/1605.06391/) (2016)
- [TensorNetwork for Machine Learning](https://mlsanity.com/paper/1906.06329/) (2019)
- [Compact Trilinear Interaction for Visual Question Answering](https://mlsanity.com/paper/1909.11874/) (2019)
- [Quantum Language Model with Entanglement Embedding for Question Answering](https://mlsanity.com/paper/2008.09943/) (2020)
- [Polynomial Networks and Factorization Machines: New Insights and Efficient Training Algorithms](https://mlsanity.com/paper/1607.08810/) (2016)
- [Frankenstein: Learning Deep Face Representations using Small Data](https://mlsanity.com/paper/1603.06470/) (2016)

## Built on by (most cited first, 3 of 3)

- [Fourier Series Guided Design of Quantum Convolutional Neural Networks for Enhanced Time Series Forecasting](https://mlsanity.com/paper/2404.15377/) (2024)
- [Selective Feature Re-Encoded Quantum Convolutional Neural Network with Joint Optimization for Image Classification](https://mlsanity.com/paper/2507.02086/) (2025)
- [QiNN-QJ: A Quantum-inspired Neural Network with Quantum Jump for Multimodal Sentiment Analysis](https://mlsanity.com/paper/2510.27091/) (2025)

## Abstract (arXiv)

States of quantum many-body systems are defined in a high-dimensional Hilbert space, where rich and complex interactions among subsystems can be modelled. In machine learning, complex multiple multilinear correlations may also exist within input features. In this paper, we present a quantum-inspired multilinear model, named Residual Tensor Train (ResTT), to capture the multiple multilinear correlations of features, from low to high orders, within a single model. ResTT is able to build a robust decision boundary in a high-dimensional space for solving fitting and classification tasks. In particular, we prove that the fully-connected layer and the Volterra series can be taken as special cases of ResTT. Furthermore, we derive the rule for weight initialization that stabilizes the training of ResTT based on a mean-field analysis. We prove that such a rule is much more relaxed than that of TT, which means ResTT can easily address the vanishing and exploding gradient problem that exists in the existing TT models. Numerical experiments demonstrate that ResTT outperforms the state-of-the-art tensor network and benchmark deep learning models on MNIST and Fashion-MNIST datasets. Moreover, ResTT achieves better performance than other statistical methods on two practical examples with limited data which are known to have complex feature interactions.