# Towards Truly Unsupervised Evaluation of Feature Selection -- Extended Version

arXiv: https://arxiv.org/abs/2608.12057 · HTML: https://mlsanity.com/paper/2608.12057/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 9 of its 39 references that are in the map.
- First posted on arXiv 12 Aug 2026.

## Builds on (most cited first, 9 of 9)

- [Sinkhorn Distances: Lightspeed Computation of Optimal Transportation Distances](https://mlsanity.com/paper/1306.0895/) (2013)
- [Computational Optimal Transport](https://mlsanity.com/paper/1803.00567/) (2018)
- [Lightspeed Geometric Dataset Distance via Sliced Optimal Transport](https://mlsanity.com/paper/2501.18901/) (2025)
- [Unsupervised Feature Selection based on Adaptive Similarity Learning and Subspace Clustering](https://mlsanity.com/paper/1912.05458/) (2019)
- [Metrics for Inter-Dataset Similarity with Example Applications in Synthetic Data and Feature Selection Evaluation -- Extended Version](https://mlsanity.com/paper/2501.09591/) (2025)
- [FSEVAL: Feature Selection Evaluation Toolbox and Dashboard](https://mlsanity.com/paper/2604.18227/) (2026)
- [Randomized PCA Forest for Unsupervised Outlier Detection](https://mlsanity.com/paper/2508.12776/) (2025)
- [MARS: Magnitude-Aware Rank Statistics](https://mlsanity.com/paper/2605.23563/) (2026)
- [Worse than Random: The Importance of a Baseline for Unsupervised Feature Selection](https://mlsanity.com/paper/2605.22973/) (2026)

## Abstract (arXiv)

Feature selection is one of the most important and fundamental tasks in data mining, tackled by a family of methods with an established set of evaluation techniques to measure the quality of a specific method. Most of the methods commonly used for the unsupervised evaluation of feature selection algorithms suffer from critical design flaws which question their unsupervised nature. In this paper, we provide a critical discussion on the established allegedly unsupervised evaluation techniques, and shed light on the reasons why they are not truly unsupervised but, at best, supervised evaluation under an unsupervised downstream task. We also propose a novel, truly unsupervised evaluation framework to measure the quality of the feature selection algorithms without any form of information about the labels. The proposed framework utilizes unsupervised Principal Component Analysis, and optimal transport to measure the quality of the feature selection methods in a truly unsupervised manner.