# Factor Augmented Tensor-on-Tensor Neural Networks

arXiv: https://arxiv.org/abs/2405.19610 · HTML: https://mlsanity.com/paper/2405.19610/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 41 references that are in the map.
- First posted on arXiv 30 May 2024.

## Builds on (most cited first, 23 of 23)

- [An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling](https://mlsanity.com/paper/1803.01271/) (2018)
- [Think Globally, Act Locally: A Deep Neural Network Approach to High-Dimensional Time Series Forecasting](https://mlsanity.com/paper/1905.03806/) (2019)
- [Tensor Regression Networks](https://mlsanity.com/paper/1707.08308/) (2017)
- [Factor Augmented Sparse Throughput Deep ReLU Neural Networks for High Dimensional Regression](https://mlsanity.com/paper/2210.02002/) (2022)
- [Multilinear tensor regression for longitudinal relational data](https://mlsanity.com/paper/1412.0048/) (2014)
- [Tucker Tensor Regression and Neuroimaging Analysis](https://mlsanity.com/paper/1304.5637/) (2013)
- [Tensor-on-tensor regression](https://mlsanity.com/paper/1701.01037/) (2017)
- [Parsimonious Tensor Response Regression](https://mlsanity.com/paper/1501.07815/) (2015)
- [Factor Models for High-Dimensional Tensor Time Series](https://mlsanity.com/paper/1905.07530/) (2019)
- [Tensor-on-Tensor Regression: Riemannian Optimization, Over-parameterization, Statistical-computational Gap, and Their Interplay](https://mlsanity.com/paper/2206.08756/) (2022)
- [A novel approach for fusion of heterogeneous sources of data](https://mlsanity.com/paper/1803.00138/) (2018)
- [Sufficient Forecasting Using Factor Models](https://mlsanity.com/paper/1505.07414/) (2015)
- [STORE: Sparse Tensor Response Regression and Neuroimaging Analysis](https://mlsanity.com/paper/1609.04523/) (2016)
- [Tensor Regression Using Low-rank and Sparse Tucker Decompositions](https://mlsanity.com/paper/1911.03725/) (2019)
- [CP Factor Model for Dynamic Tensors](https://mlsanity.com/paper/2110.15517/) (2021)
- [Inverse Moment Methods for Sufficient Forecasting using High-Dimensional Predictors](https://mlsanity.com/paper/1705.00395/) (2017)
- [Tensor Representation in High-Frequency Financial Data for Price Change Prediction](https://mlsanity.com/paper/1709.01268/) (2017)
- [Tensor Factor Model Estimation by Iterative Projection](https://mlsanity.com/paper/2006.02611/) (2020)
- [DFacTo: Distributed Factorization of Tensors](https://mlsanity.com/paper/1406.4519/) (2014)
- [Rank Determination in Tensor Factor Model](https://mlsanity.com/paper/2011.07131/) (2020)
- [TENT: Tensorized Encoder Transformer for Temperature Forecasting](https://mlsanity.com/paper/2106.14742/) (2021)
- [Tensor Principal Component Analysis in High Dimensional CP Models](https://mlsanity.com/paper/2108.04428/) (2021)
- [Theoretical and Experimental Analyses of Tensor-Based Regression and Classification](https://mlsanity.com/paper/1509.01770/) (2015)

## Built on by (most cited first, 7 of 7)

- [Covariate-Adjusted Deep Causal Learning for Heterogeneous Panel Data Models](https://mlsanity.com/paper/2505.20536/) (2025)
- [Factor Informed Double Deep Learning For Average Treatment Effect Estimation](https://mlsanity.com/paper/2508.17136/) (2025)
- [Tensor-on-tensor Regression Neural Networks for Process Modeling with High-dimensional Data](https://mlsanity.com/paper/2510.05329/) (2025)
- [TEAFormers: TEnsor-Augmented Transformers for Multi-Dimensional Time Series Forecasting](https://mlsanity.com/paper/2410.20439/) (2024)
- [Supervised Dynamic Dimension Reduction with Deep Neural Network](https://mlsanity.com/paper/2508.03546/) (2025)
- [Factor Augmented Supervised Learning with Text Embeddings](https://mlsanity.com/paper/2508.06548/) (2025)
- [Factor Augmented High-Dimensional SGD](https://mlsanity.com/paper/2605.19291/) (2026)

## Abstract (arXiv)

This paper studies the prediction task of tensor-on-tensor regression in which both covariates and responses are multi-dimensional arrays (a.k.a., tensors) across time with arbitrary tensor order and data dimension. Existing methods either focused on linear models without accounting for possibly nonlinear relationships between covariates and responses, or directly employed black-box deep learning algorithms that failed to utilize the inherent tensor structure. In this work, we propose a Factor Augmented Tensor-on-Tensor Neural Network (FATTNN) that integrates tensor factor models into deep neural networks. We begin with summarizing and extracting useful predictive information (represented by the ``factor tensor'') from the complex structured tensor covariates, and then proceed with the prediction task using the estimated factor tensor as input of a temporal convolutional neural network. The proposed methods effectively handle nonlinearity between complex data structures, and improve over traditional statistical models and conventional deep learning approaches in both prediction accuracy and computational cost. By leveraging tensor factor models, our proposed methods exploit the underlying latent factor structure to enhance the prediction, and in the meantime, drastically reduce the data dimensionality that speeds up the computation. The empirical performances of our proposed methods are demonstrated via simulation studies and real-world applications to three public datasets. Numerical results show that our proposed algorithms achieve substantial increases in prediction accuracy and significant reductions in computational time compared to benchmark methods.