# Nonlinear Multiple Response Regression and Learning of Latent Spaces

arXiv: https://arxiv.org/abs/2503.21608 · HTML: https://mlsanity.com/paper/2503.21608/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 45 references that are in the map.
- First posted on arXiv 27 Mar 2025.

## Builds on (most cited first, 14 of 14)

- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [Sliced Score Matching: A Scalable Approach to Density and Score Estimation](https://mlsanity.com/paper/1905.07088/) (2019)
- [k-Sparse Autoencoders](https://mlsanity.com/paper/1312.5663/) (2013)
- [Neural Networks can Learn Representations with Gradient Descent](https://mlsanity.com/paper/2206.15144/) (2022)
- [Estimating High Order Gradients of the Data Distribution by Denoising](https://mlsanity.com/paper/2111.04726/) (2021)
- [Neural Networks Efficiently Learn Low-Dimensional Representations with SGD](https://mlsanity.com/paper/2209.14863/) (2022)
- [Gradient Estimators for Implicit Models](https://mlsanity.com/paper/1705.07107/) (2017)
- [A Spectral Approach to Gradient Estimation for Implicit Distributions](https://mlsanity.com/paper/1806.02925/) (2018)
- [Nonparametric Score Estimators](https://mlsanity.com/paper/2005.10099/) (2020)
- [Gradient-free Hamiltonian Monte Carlo with Efficient Kernel Exponential Families](https://mlsanity.com/paper/1506.02564/) (2015)
- [A Blockwise Descent Algorithm for Group-penalized Multiresponse and Multinomial Regression](https://mlsanity.com/paper/1311.6529/) (2013)
- [Score Function Features for Discriminative Learning: Matrix and Tensor Framework](https://mlsanity.com/paper/1412.2863/) (2014)
- [A convex formulation for high-dimensional sparse sliced inverse regression](https://mlsanity.com/paper/1809.06024/) (2018)
- [Tensor Methods for Additive Index Models under Discordance and Heterogeneity](https://mlsanity.com/paper/1807.06693/) (2018)

## Built on by (most cited first, 1 of 1)

- [Multi-Source Transfer Learning of Sparse Single-Index Models](https://mlsanity.com/paper/2606.29658/) (2026)

## Abstract (arXiv)

Identifying low-dimensional latent structures within high-dimensional data has long been a central topic in the machine learning community, driven by the need for data compression, storage, transmission, and deeper data understanding. Traditional methods, such as principal component analysis (PCA) and autoencoders (AE), operate in an unsupervised manner, ignoring label information even when it is available. In this work, we introduce a unified method capable of learning latent spaces in both unsupervised and supervised settings. We formulate the problem as a nonlinear multiple-response regression within an index model context. By applying the generalized Stein's lemma, the latent space can be estimated without knowing the nonlinear link functions. Our method can be viewed as a nonlinear generalization of PCA. Moreover, unlike AE and other neural network methods that operate as "black boxes", our approach not only offers better interpretability but also reduces computational complexity while providing strong theoretical guarantees. Comprehensive numerical experiments and real data analyses demonstrate the superior performance of our method.