# Stable Tensor Neural Networks for Rapid Deep Learning

arXiv: https://arxiv.org/abs/1811.06569 · HTML: https://mlsanity.com/paper/1811.06569/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Nov 2018.

## Built on by (most cited first, 6 of 6)

- [Transformed Low-Rank Parameterization Can Help Robust Generalization for Tensor Neural Networks](https://mlsanity.com/paper/2303.00196/) (2023)
- [Optimal Matrix-Mimetic Tensor Algebras via Variable Projection](https://mlsanity.com/paper/2406.06942/) (2024)
- [T-ADAF: Adaptive Data Augmentation Framework for Image Classification Network based on Tensor T-product Operator](https://mlsanity.com/paper/2306.04240/) (2023)
- [Quantile-Based Randomized Kaczmarz for Corrupted Tensor Linear Systems](https://mlsanity.com/paper/2503.18190/) (2025)
- [Real-Time Network Traffic Forecasting with Missing Data: A Generative Model Approach](https://mlsanity.com/paper/2506.09647/) (2025)
- [Data Understanding Survey: Pursuing Improved Dataset Characterization Via Tensor-based Methods](https://mlsanity.com/paper/2510.14161/) (2025)

## Abstract (arXiv)

We propose a tensor neural network (t-NN) framework that offers an exciting new paradigm for designing neural networks with multidimensional (tensor) data. Our network architecture is based on the t-product (Kilmer and Martin, 2011), an algebraic formulation to multiply tensors via circulant convolution. In this t-product algebra, we interpret tensors as t-linear operators analogous to matrices as linear operators, and hence our framework inherits mimetic matrix properties. To exemplify the elegant, matrix-mimetic algebraic structure of our t-NNs, we expand on recent work (Haber and Ruthotto, 2017) which interprets deep neural networks as discretizations of non-linear differential equations and introduces stable neural networks which promote superior generalization. Motivated by this dynamic framework, we introduce a stable t-NN which facilitates more rapid learning because of its reduced, more powerful parameterization. Through our high-dimensional design, we create a more compact parameter space and extract multidimensional correlations otherwise latent in traditional algorithms. We further generalize our t-NN framework to a family of tensor-tensor products (Kernfeld, Kilmer, and Aeron, 2015) which still induce a matrix-mimetic algebraic structure. Through numerical experiments on the MNIST and CIFAR-10 datasets, we demonstrate the more powerful parameterizations and improved generalizability of stable t-NNs.