# Feature Selection in the Contrastive Analysis Setting

arXiv: https://arxiv.org/abs/2310.18531 · HTML: https://mlsanity.com/paper/2310.18531/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 54 references that are in the map.
- First posted on arXiv 27 Oct 2023.

## Builds on (most cited first, 17 of 17)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016)
- [The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables](https://mlsanity.com/paper/1611.00712/) (2016)
- [A Theoretical Analysis of Contrastive Unsupervised Representation Learning](https://mlsanity.com/paper/1902.09229/) (2019)
- [Concrete Autoencoders for Differentiable Feature Selection and Reconstruction](https://mlsanity.com/paper/1901.09346/) (2019)
- [LassoNet: A Neural Network with Feature Sparsity](https://mlsanity.com/paper/1907.12207/) (2019)
- [Learning to Maximize Mutual Information for Dynamic Feature Selection](https://mlsanity.com/paper/2301.00557/) (2023)
- [Sparse-Input Neural Networks for High-dimensional Nonparametric Regression and Classification](https://mlsanity.com/paper/1711.07592/) (2017)
- [Contrastive Variational Autoencoder Enhances Salient Features](https://mlsanity.com/paper/1902.04601/) (2019)
- [Unsupervised learning with contrastive latent variable models](https://mlsanity.com/paper/1811.06094/) (2018)
- [Moment Matching Deep Contrastive Latent Variable Models](https://mlsanity.com/paper/2202.10560/) (2022)
- [Probabilistic Contrastive Principal Component Analysis](https://mlsanity.com/paper/2012.07977/) (2020)
- [Contrastive latent variable modeling with application to case-control sequencing experiments](https://mlsanity.com/paper/2102.06731/) (2021)
- [Learning Disentangled Representations with Reference-Based Variational Autoencoders](https://mlsanity.com/paper/1901.08534/) (2019)

## Built on by (most cited first, 3 of 3)

- [Partial Information Decomposition for Data Interpretability and Feature Selection](https://mlsanity.com/paper/2405.19212/) (2024)
- [Contrastive Dimension Reduction: A Systematic Review](https://mlsanity.com/paper/2510.11847/) (2025)
- [FedHPro: Federated Hyper-Prototype Learning via Gradient Matching](https://mlsanity.com/paper/2605.13475/) (2026)

## Abstract (arXiv)

Contrastive analysis (CA) refers to the exploration of variations uniquely enriched in a target dataset as compared to a corresponding background dataset generated from sources of variation that are irrelevant to a given task. For example, a biomedical data analyst may wish to find a small set of genes to use as a proxy for variations in genomic data only present among patients with a given disease (target) as opposed to healthy control subjects (background). However, as of yet the problem of feature selection in the CA setting has received little attention from the machine learning community. In this work we present contrastive feature selection (CFS), a method for performing feature selection in the CA setting. We motivate our approach with a novel information-theoretic analysis of representation learning in the CA setting, and we empirically validate CFS on a semi-synthetic dataset and four real-world biomedical datasets. We find that our method consistently outperforms previously proposed state-of-the-art supervised and fully unsupervised feature selection methods not designed for the CA setting. An open-source implementation of our method is available at https://github.com/suinleelab/CFS.