# Tensor Regression Networks with various Low-Rank Tensor Approximations

arXiv: https://arxiv.org/abs/1712.09520 · HTML: https://mlsanity.com/paper/1712.09520/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Dec 2017.

## Built on by (most cited first, 8 of 8)

- [Tensor Methods in Computer Vision and Deep Learning](https://mlsanity.com/paper/2107.03436/) (2021)
- [High-Order Pooling for Graph Neural Networks with Tensor Decomposition](https://mlsanity.com/paper/2205.11691/) (2022)
- [tn4ml: Tensor Network Training and Customization for Machine Learning](https://mlsanity.com/paper/2502.13090/) (2025)
- [Semi-tensor Product-based TensorDecomposition for Neural Network Compression](https://mlsanity.com/paper/2109.15200/) (2021)
- [Efficient Tensor Contraction via Fast Count Sketch](https://mlsanity.com/paper/2106.13062/) (2021)
- [Learning From Graph-Structured Data: Addressing Design Issues and Exploring Practical Applications in Graph Representation Learning](https://mlsanity.com/paper/2411.07269/) (2024)
- [MELCOT: A Hybrid Learning Architecture with Marginal Preservation for Matrix-Valued Regression](https://mlsanity.com/paper/2509.23315/) (2025)
- [TinyCardioUNet: IMU-to-ECG Translation with Graph-Encoded Inter-Axis Dependencies and Tensor Decomposition-Based Parameter Reduction](https://mlsanity.com/paper/2609.29322/) (2026)

## Abstract (arXiv)

Tensor regression networks achieve high compression rate of neural networks while having slight impact on performances. They do so by imposing low tensor rank structure on the weight matrices of fully connected layers. In recent years, tensor regression networks have been investigated from the perspective of their compressive power, however, the regularization effect of enforcing low-rank tensor structure has not been investigated enough. We study tensor regression networks using various low-rank tensor approximations, aiming to compare the compressive and regularization power of different low-rank constraints. We evaluate the compressive and regularization performances of the proposed model with both deep and shallow convolutional neural networks. The outcome of our experiment suggests the superiority of Global Average Pooling Layer over Tensor Regression Layer when applied to deep convolutional neural network with CIFAR-10 dataset. On the contrary, shallow convolutional neural networks with tensor regression layer and dropout achieved lower test error than both Global Average Pooling and fully-connected layer with dropout function when trained with a small number of samples.