# Contrastive inverse regression for dimension reduction

arXiv: https://arxiv.org/abs/2305.12287 · HTML: https://mlsanity.com/paper/2305.12287/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 70 references that are in the map.
- First posted on arXiv 20 May 2023.

## Builds on (most cited first, 11 of 11)

- [Global rates of convergence for nonconvex optimization on manifolds](https://mlsanity.com/paper/1605.08101/) (2016)
- [Simple Techniques Work Surprisingly Well for Neural Network Test Prioritization and Active Learning (Replicability Study)](https://mlsanity.com/paper/2205.00664/) (2022)
- [Multivariate Gaussian and Student-t Process Regression for Multi-output Prediction](https://mlsanity.com/paper/1703.04455/) (2017)
- [Supervised Dimensionality Reduction for Big Data](https://mlsanity.com/paper/1709.01233/) (2017)
- [Efficient Manifold and Subspace Approximations with Spherelets](https://mlsanity.com/paper/1706.08263/) (2017)
- [Probabilistic Contrastive Principal Component Analysis](https://mlsanity.com/paper/2012.07977/) (2020)
- [Contrastive latent variable modeling with application to case-control sequencing experiments](https://mlsanity.com/paper/2102.06731/) (2021)
- [A Distance-preserving Matrix Sketch](https://mlsanity.com/paper/2009.03979/) (2020)
- [Variable selection for general index models via sliced inverse regression](https://mlsanity.com/paper/1304.4056/) (2013)
- [Spherical Rotation Dimension Reduction with Geometric Loss Functions](https://mlsanity.com/paper/2204.10975/) (2022)
- [Nonparametric Multi-shape Modeling with Uncertainty Quantification](https://mlsanity.com/paper/2206.09127/) (2022)

## Built on by (most cited first, 2 of 2)

- [Efficient Decision Trees for Tensor Regressions](https://mlsanity.com/paper/2408.01926/) (2024)
- [Contrastive Dimension Reduction: A Systematic Review](https://mlsanity.com/paper/2510.11847/) (2025)

## Abstract (arXiv)

Supervised dimension reduction (SDR) has been a topic of growing interest in data science, as it enables the reduction of high-dimensional covariates while preserving the functional relation with certain response variables of interest. However, existing SDR methods are not suitable for analyzing datasets collected from case-control studies. In this setting, the goal is to learn and exploit the low-dimensional structure unique to or enriched by the case group, also known as the foreground group. While some unsupervised techniques such as the contrastive latent variable model and its variants have been developed for this purpose, they fail to preserve the functional relationship between the dimension-reduced covariates and the response variable. In this paper, we propose a supervised dimension reduction method called contrastive inverse regression (CIR) specifically designed for the contrastive setting. CIR introduces an optimization problem defined on the Stiefel manifold with a non-standard loss function. We prove the convergence of CIR to a local optimum using a gradient descent-based algorithm, and our numerical study empirically demonstrates the improved performance over competing methods for high-dimensional data.