# Deep Unsupervised Feature Selection by Discarding Nuisance and Correlated Features

arXiv: https://arxiv.org/abs/2110.05306 · HTML: https://mlsanity.com/paper/2110.05306/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 29 references that are in the map.
- First posted on arXiv 11 Oct 2021.

## Builds on (most cited first, 4 of 4)

- [The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables](https://mlsanity.com/paper/1611.00712/) (2016)
- [SpectralNet: Spectral Clustering using Deep Neural Networks](https://mlsanity.com/paper/1801.01587/) (2018)
- [Concrete Autoencoders for Differentiable Feature Selection and Reconstruction](https://mlsanity.com/paper/1901.09346/) (2019)
- [Differentiable Unsupervised Feature Selection based on a Gated Laplacian](https://mlsanity.com/paper/2007.04728/) (2020)

## Built on by (most cited first, 11 of 11)

- [Interpretable Deep Clustering for Tabular Data](https://mlsanity.com/paper/2306.04785/) (2023)
- [Domain-Generalizable Multiple-Domain Clustering](https://mlsanity.com/paper/2301.13530/) (2023)
- [Supervised Feature Selection with Neuron Evolution in Sparse Neural Networks](https://mlsanity.com/paper/2303.07200/) (2023)
- [Contextual Feature Selection with Conditional Stochastic Gates](https://mlsanity.com/paper/2312.14254/) (2023)
- [Multi-modal Differentiable Unsupervised Feature Selection](https://mlsanity.com/paper/2303.09381/) (2023)
- [Hybrid Autoencoders for Tabular Data: Leveraging Model-Based Augmentation in Low-Label Settings](https://mlsanity.com/paper/2511.06961/) (2025)
- [Spectral Self-supervised Feature Selection](https://mlsanity.com/paper/2407.09061/) (2024)
- [Neuronal Cell Type Classification using Deep Learning](https://mlsanity.com/paper/2306.00528/) (2023)
- [Kernel Alignment for Unsupervised Feature Selection via Matrix Factorization](https://mlsanity.com/paper/2403.14688/) (2024)
- [Scalable unsupervised feature selection via weight stability](https://mlsanity.com/paper/2506.06114/) (2025)
- [When Features Become Instances: Inverted Contrastive Learning for Unsupervised Feature Selection](https://mlsanity.com/paper/2609.00782/) (2026)

## Abstract (arXiv)

Modern datasets often contain large subsets of correlated features and nuisance features, which are not or loosely related to the main underlying structures of the data. Nuisance features can be identified using the Laplacian score criterion, which evaluates the importance of a given feature via its consistency with the Graph Laplacians' leading eigenvectors. We demonstrate that in the presence of large numbers of nuisance features, the Laplacian must be computed on the subset of selected features rather than on the complete feature set. To do this, we propose a fully differentiable approach for unsupervised feature selection, utilizing the Laplacian score criterion to avoid the selection of nuisance features. We employ an autoencoder architecture to cope with correlated features, trained to reconstruct the data from the subset of selected features. Building on the recently proposed concrete layer that allows controlling for the number of selected features via architectural design, simplifying the optimization process. Experimenting on several real-world datasets, we demonstrate that our proposed approach outperforms similar approaches designed to avoid only correlated or nuisance features, but not both. Several state-of-the-art clustering results are reported.