# Deep Dimension Reduction for Supervised Representation Learning

arXiv: https://arxiv.org/abs/2006.05865 · HTML: https://mlsanity.com/paper/2006.05865/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Jun 2020.

## Built on by (most cited first, 15 of 15)

- [Doubly Robust Conditional Independence Testing with Generative Neural Networks](https://mlsanity.com/paper/2407.17694/) (2024)
- [Deep Transfer Learning: Model Framework and Error Analysis](https://mlsanity.com/paper/2410.09383/) (2024)
- [Belted and Ensembled Neural Network for Linear and Nonlinear Sufficient Dimension Reduction](https://mlsanity.com/paper/2412.08961/) (2024)
- [On Conditional Stochastic Interpolation for Generative Nonlinear Sufficient Dimension Reduction](https://mlsanity.com/paper/2512.18971/) (2025)
- [Privacy-Preserving Machine Learning for Collaborative Data Sharing via Auto-encoder Latent Space Embeddings](https://mlsanity.com/paper/2211.05717/) (2022)
- [From Conditional to Unconditional Independence: Testing Conditional Independence via Transport Maps](https://mlsanity.com/paper/2504.09567/) (2025)
- [Fr'echet Cumulative Covariance Net for Deep Nonlinear Sufficient Dimension Reduction with Random Objects](https://mlsanity.com/paper/2502.15374/) (2025)
- [Transfer Learning through Enhanced Sufficient Representation: Enriching Source Domain Knowledge with Target Data](https://mlsanity.com/paper/2502.20414/) (2025)
- [DeepSuM: Deep Sufficient Modality Learning Framework](https://mlsanity.com/paper/2503.01728/) (2025)
- [A general framework for adaptive nonparametric dimensionality reduction](https://mlsanity.com/paper/2511.09486/) (2025)
- [Online Nonparametric Supervised Learning for Massive Data](https://mlsanity.com/paper/2405.19486/) (2024)
- [Fair Sufficient Representation Learning](https://mlsanity.com/paper/2504.01030/) (2025)
- [Semi-Supervised Conditional Generative Learning through Stochastic Interpolation and Sufficient Representations](https://mlsanity.com/paper/2607.16725/) (2026)
- [Latent-Space Importance Weighting under Covariate Shift](https://mlsanity.com/paper/2610.15571/) (2026)
- [Testing Conditional Distribution Equality Using Generative Models](https://mlsanity.com/paper/2610.20447/) (2026)

## Abstract (arXiv)

The goal of supervised representation learning is to construct effective data representations for prediction. Among all the characteristics of an ideal nonparametric representation of high-dimensional complex data, sufficiency, low dimensionality and disentanglement are some of the most essential ones. We propose a deep dimension reduction approach to learning representations with these characteristics. The proposed approach is a nonparametric generalization of the sufficient dimension reduction method. We formulate the ideal representation learning task as that of finding a nonparametric representation that minimizes an objective function characterizing conditional independence and promoting disentanglement at the population level. We then estimate the target representation at the sample level nonparametrically using deep neural networks. We show that the estimated deep nonparametric representation is consistent in the sense that its excess risk converges to zero. Our extensive numerical experiments using simulated and real benchmark data demonstrate that the proposed methods have better performance than several existing dimension reduction methods and the standard deep learning models in the context of classification and regression.