# Deep Learning Partial Least Squares

arXiv: https://arxiv.org/abs/2106.14085 · HTML: https://mlsanity.com/paper/2106.14085/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Jun 2021.

## Built on by (most cited first, 5 of 5)

- [Merging Two Cultures: Deep and Statistical Learning](https://mlsanity.com/paper/2110.11561/) (2021)
- [Quantum Bayesian Computation](https://mlsanity.com/paper/2208.08068/) (2022)
- [Generative Modeling: A Review](https://mlsanity.com/paper/2501.05458/) (2024)
- [Deep Partial Least Squares for Empirical Asset Pricing](https://mlsanity.com/paper/2206.10014/) (2022)
- [Generative Bayesian Computation as a Scalable Alternative to Gaussian Process Surrogates](https://mlsanity.com/paper/2602.21408/) (2026)

## Abstract (arXiv)

High dimensional data reduction techniques are provided by using partial least squares within deep learning. Our framework provides a nonlinear extension of PLS together with a disciplined approach to feature selection and architecture design in deep learning. This leads to a statistical interpretation of deep learning that is tailor made for predictive problems. We can use the tools of PLS, such as scree-plot, bi-plot to provide model diagnostics. Posterior predictive uncertainty is available using MCMC methods at the last layer. Thus we achieve the best of both worlds: scalability and fast predictive rule construction together with uncertainty quantification. Our key construct is to employ deep learning within PLS by predicting the output scores as a deep learner of the input scores. As with PLS our X-scores are constructed using SVD and applied to both regression and classification problems and are fast and scalable. Following Frank and Friedman 1993, we provide a Bayesian shrinkage interpretation of our nonlinear predictor. We introduce a variety of new partial least squares models: PLS-ReLU, PLS-Autoencoder, PLS-Trees and PLS-GP. To illustrate our methodology, we use simulated examples and the analysis of preferences of orange juice and predicting wine quality as a function of input characteristics. We also illustrate Brillinger's estimation procedure to provide the feature selection and data dimension reduction. Finally, we conclude with directions for future research.