# Pre-processing with Orthogonal Decompositions for High-dimensional Explanatory Variables

arXiv: https://arxiv.org/abs/2106.09071 · HTML: https://mlsanity.com/paper/2106.09071/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 10 of its 49 references that are in the map.
- First posted on arXiv 16 Jun 2021.

## Builds on (most cited first, 10 of 10)

- [Challenges of Big Data Analysis](https://mlsanity.com/paper/1308.1479/) (2013)
- [Tuning parameter selection in high dimensional penalized likelihood](https://mlsanity.com/paper/1605.03321/) (2016)
- [Spectral Deconfounding via Perturbed Sparse Linear Models](https://mlsanity.com/paper/1811.05352/) (2018)
- [Factor-Adjusted Regularized Model Selection](https://mlsanity.com/paper/1612.08490/) (2016)
- [Model Selection for High Dimensional Quadratic Regression via Regularization](https://mlsanity.com/paper/1501.00049/) (2014)
- [A lava attack on the recovery of sums of dense and sparse signals](https://mlsanity.com/paper/1502.03155/) (2015)
- [High dimensional thresholded regression and shrinkage effect](https://mlsanity.com/paper/1605.03306/) (2016)
- [DECOrrelated feature space partitioning for distributed sparse regression](https://mlsanity.com/paper/1602.02575/) (2016)
- [Guarding against Spurious Discoveries in High Dimensions](https://mlsanity.com/paper/1512.01617/) (2015)
- [Likelihood Adaptively Modified Penalties](https://mlsanity.com/paper/1308.5036/) (2013)

## Abstract (arXiv)

Strong correlations between explanatory variables are problematic for high-dimensional regularized regression methods. Due to the violation of the Irrepresentable Condition, the popular LASSO method may suffer from false inclusions of inactive variables. In this paper, we propose pre-processing with orthogonal decompositions (PROD) for the explanatory variables in high-dimensional regressions. The PROD procedure is constructed based upon a generic orthogonal decomposition of the design matrix. We demonstrate by two concrete cases that the PROD approach can be effectively constructed for improving the performance of high-dimensional penalized regression. Our theoretical analysis reveals their properties and benefits for high-dimensional penalized linear regression with LASSO. Extensive numerical studies with simulations and data analysis show the promising performance of the PROD.