# Asymptotic Optimism for Tensor Regression Models with Applications to Neural Network Compression

arXiv: https://arxiv.org/abs/2603.26048 · HTML: https://mlsanity.com/paper/2603.26048/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 8 of its 54 references that are in the map.
- First posted on arXiv 27 Mar 2026.

## Builds on (most cited first, 8 of 8)

- [AlphaLoRA: Assigning LoRA Experts Based on Layer Training Quality](https://mlsanity.com/paper/2410.10054/) (2024)
- [Sharded Bayesian Additive Regression Trees](https://mlsanity.com/paper/2306.00361/) (2023)
- [Asymptotic Optimism of Random-Design Linear and Kernel Regression Models](https://mlsanity.com/paper/2502.12999/) (2025)
- [Wedge Sampling: Efficient Tensor Completion with Nearly-Linear Sample Complexity](https://mlsanity.com/paper/2602.05869/) (2026)
- [An efficient tensor regression for high-dimensional data](https://mlsanity.com/paper/2205.13734/) (2022)
- [Frontal Slice Approaches for Tensor Linear Systems](https://mlsanity.com/paper/2408.13547/) (2024)
- [Wavelet Tree Ensembles for Triangulable Manifolds](https://mlsanity.com/paper/2601.20254/) (2026)
- [Tucker tensor factor models: matricization and mode-wise PCA estimation](https://mlsanity.com/paper/2206.02508/) (2022)

## Abstract (arXiv)

We study rank selection for low-rank tensor regression under random covariates design. Under a Gaussian random-design model and some mild conditions, we derive population expressions for the expected training-testing discrepancy (optimism) for both CP and Tucker decomposition. We further demonstrate that the optimism is minimized at the true tensor rank for both CP and Tucker regression. This yields a prediction-oriented rank-selection rule that aligns with cross-validation and extends naturally to tensor-model averaging. We also discuss conditions under which under- or over-ranked models may appear preferable, thereby clarifying the scope of the method. Finally, we showcase its practical utility on a real-world image regression task and extend its application to tensor-based compression of neural network, highlighting its potential for model selection in deep learning.