# Structured Low-Rank Tensors for Generalized Linear Models

arXiv: https://arxiv.org/abs/2308.02922 · HTML: https://mlsanity.com/paper/2308.02922/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 58 references that are in the map.
- First posted on arXiv 5 Aug 2023.

## Builds on (most cited first, 16 of 16)

- [MedMNIST Classification Decathlon: A Lightweight AutoML Benchmark for Medical Image Analysis](https://mlsanity.com/paper/2010.14925/) (2020)
- [Logistic Regression: The Importance of Being Improper](https://mlsanity.com/paper/1803.09349/) (2018)
- [IntrA: 3D Intracranial Aneurysm Dataset for Deep Learning](https://mlsanity.com/paper/2003.02920/) (2020)
- [Tucker Tensor Regression and Neuroimaging Analysis](https://mlsanity.com/paper/1304.5637/) (2013)
- [High-dimensional classification by sparse logistic regression](https://mlsanity.com/paper/1706.08344/) (2017)
- [Covariance Estimation in High Dimensions via Kronecker Product Expansions](https://mlsanity.com/paper/1302.2686/) (2013)
- [ISLET: Fast and Optimal Low-rank Tensor Regression via Importance Sketching](https://mlsanity.com/paper/1911.03804/) (2019)
- [Computing Large-Scale Matrix and Tensor Decomposition with Structured Factors: A Unified Nonconvex Optimization Perspective](https://mlsanity.com/paper/2006.08183/) (2020)
- [Tensor Regression Using Low-rank and Sparse Tucker Decompositions](https://mlsanity.com/paper/1911.03725/) (2019)
- [Sparse Bilinear Logistic Regression](https://mlsanity.com/paper/1404.4104/) (2014)
- [Model selection and minimax estimation in generalized linear models](https://mlsanity.com/paper/1409.8491/) (2014)
- [On the Minimax Risk of Dictionary Learning](https://mlsanity.com/paper/1507.05498/) (2015)
- [Learning Product Graphs Underlying Smooth Graph Signals](https://mlsanity.com/paper/2002.11277/) (2020)
- [Minimax Lower Bounds for Kronecker-Structured Dictionary Learning](https://mlsanity.com/paper/1605.05284/) (2016)
- [Learning Mixtures of Separable Dictionaries for Tensor Data: Analysis and Algorithms](https://mlsanity.com/paper/1903.09284/) (2019)
- [A Minimax Lower Bound for Low-Rank Matrix-Variate Logistic Regression](https://mlsanity.com/paper/2105.14673/) (2021)

## Built on by (most cited first, 1 of 1)

- [A Muon-Accelerated Algorithm for Low Separation Rank Tensor Generalized Linear Models](https://mlsanity.com/paper/2604.04726/) (2026)

## Abstract (arXiv)

Recent works have shown that imposing tensor structures on the coefficient tensor in regression problems can lead to more reliable parameter estimation and lower sample complexity compared to vector-based methods. This work investigates a new low-rank tensor model, called Low Separation Rank (LSR), in Generalized Linear Model (GLM) problems. The LSR model -- which generalizes the well-known Tucker and CANDECOMP/PARAFAC (CP) models, and is a special case of the Block Tensor Decomposition (BTD) model -- is imposed onto the coefficient tensor in the GLM model. This work proposes a block coordinate descent algorithm for parameter estimation in LSR-structured tensor GLMs. Most importantly, it derives a minimax lower bound on the error threshold on estimating the coefficient tensor in LSR tensor GLM problems. The minimax bound is proportional to the intrinsic degrees of freedom in the LSR tensor GLM problem, suggesting that its sample complexity may be significantly lower than that of vectorized GLMs. This result can also be specialised to lower bound the estimation error in CP and Tucker-structured GLMs. The derived bounds are comparable to tight bounds in the literature for Tucker linear regression, and the tightness of the minimax lower bound is further assessed numerically. Finally, numerical experiments on synthetic datasets demonstrate the efficacy of the proposed LSR tensor model for three regression types (linear, logistic and Poisson). Experiments on a collection of medical imaging datasets demonstrate the usefulness of the LSR model over other tensor models (Tucker and CP) on real, imbalanced data with limited available samples.