# When Features Become Instances: Inverted Contrastive Learning for Unsupervised Feature Selection

arXiv: https://arxiv.org/abs/2609.00782 · HTML: https://mlsanity.com/paper/2609.00782/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 23 references that are in the map.
- First posted on arXiv 1 Sep 2026.

## Builds on (most cited first, 11 of 11)

- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [NormFace: L2 Hypersphere Embedding for Face Verification](https://mlsanity.com/paper/1704.06369/) (2017)
- [Concrete Autoencoders for Differentiable Feature Selection and Reconstruction](https://mlsanity.com/paper/1901.09346/) (2019)
- [von Mises-Fisher Loss: An Exploration of Embedding Geometries for Supervised Learning](https://mlsanity.com/paper/2103.15718/) (2021)
- [Relief-Based Feature Selection: Introduction and Review](https://mlsanity.com/paper/1711.08421/) (2017)
- [Deep Metric Learning with Spherical Embedding](https://mlsanity.com/paper/2011.02785/) (2020)
- [Rethinking Self-Supervised Learning: Small is Beautiful](https://mlsanity.com/paper/2103.13559/) (2021)
- [Deep Unsupervised Feature Selection by Discarding Nuisance and Correlated Features](https://mlsanity.com/paper/2110.05306/) (2021)
- [On the Importance of Embedding Norms in Self-Supervised Learning](https://mlsanity.com/paper/2502.09252/) (2025)
- [Spectral Self-supervised Feature Selection](https://mlsanity.com/paper/2407.09061/) (2024)

## Abstract (arXiv)

Unsupervised feature selection seeks a compact subset of informative features without access to class labels, making feature utility difficult to define. Existing UFS methods therefore rely on indirect structural criteria, such as similarity preservation, locality, sparsity, cluster geometry, or reconstruction quality. In this paper, we instead study UFS through representation consistency and propose Inverted Contrastive Learning for Unsupervised Feature Selection (ICLFS), a feature-wise contrastive framework that reformulates UFS as a representation learning problem over features rather than samples. ICLFS first inverts the data matrix so that each feature is represented by its sample-profile vector, then constructs multiple masked positive views together with a shuffled negative view, and learns projector-space representations that remain consistent across these structured perturbations under an InfoNCE-based objective. Motivated by recent findings that cosine-based and InfoNCE-based training affect embedding norms, we use projector-space embedding magnitude as the saliency signal for ranking features. The resulting norm-based ranking is subsequently refined through Laplacian-Gated Ranking Correction, which suppresses locally redundant candidates while preserving salient ones. Extensive experiments on 12 benchmark datasets show that ICLFS achieves the best clustering accuracy on 10 datasets against both classical and neural baselines under the standard clustering-based UFS evaluation protocol, while remaining competitive on the other two. These results show that feature-wise contrastive representation consistency provides a strong and effective alternative to neighborhood, cluster, and reconstruction-based UFS formulations.