# Higher Order Reduced Rank Regression

arXiv: https://arxiv.org/abs/2503.06528 · HTML: https://mlsanity.com/paper/2503.06528/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 8 of its 39 references that are in the map.
- First posted on arXiv 9 Mar 2025.

## Builds on (most cited first, 8 of 8)

- [Manopt, a Matlab toolbox for optimization on manifolds](https://mlsanity.com/paper/1308.5200/) (2013)
- [Faster Kernel Ridge Regression Using Sketching and Preconditioning](https://mlsanity.com/paper/1611.03220/) (2016)
- [Low-rank tensor completion: a Riemannian manifold preconditioning approach](https://mlsanity.com/paper/1605.08257/) (2016)
- [Learning from Multiway Data: Simple and Efficient Tensor Regression](https://mlsanity.com/paper/1607.02535/) (2016)
- [Reduced-Rank Tensor-on-Tensor Regression and Tensor-variate Analysis of Variance](https://mlsanity.com/paper/2012.10249/) (2020)
- [Robust Reduced Rank Regression](https://mlsanity.com/paper/1509.03938/) (2015)
- [A Riemannian trust-region method for low-rank tensor completion](https://mlsanity.com/paper/1703.10019/) (2017)
- [Automatic differentiation for Riemannian optimization on low-rank matrix and tensor-train manifolds](https://mlsanity.com/paper/2103.14974/) (2021)

## Abstract (arXiv)

Reduced Rank Regression (RRR) is a widely used method for multi-response regression. However, RRR assumes a linear relationship between features and responses. While linear models are useful and often provide a good approximation, many real-world problems involve more complex relationships that cannot be adequately captured by simple linear interactions. One way to model such relationships is via multilinear transformations. This paper introduces Higher Order Reduced Rank Regression (HORRR), an extension of RRR that leverages multi-linear transformations, and as such is capable of capturing nonlinear interactions in multi-response regression. HORRR employs tensor representations for the coefficients and a Tucker decomposition to impose multilinear rank constraints as regularization akin to the rank constraints in RRR. Encoding these constraints as a manifold allows us to use Riemannian optimization to solve this HORRR problems. We theoretically and empirically analyze the use of Riemannian optimization for solving HORRR problems.