mlsanityData to 7 Oct 2026

Can an unsupervised clustering algorithm reproduce a categorization system?

arXiv 2408.10340 ↗First posted 19 Aug 2024stat.ML cs.LG q-fin.STcited by 2

Not on the map. Why?

Abstract (arXiv)

Peer analysis is a critical component of investment management, often relying on expert-provided categorization systems. These systems' consistency is questioned when they do not align with cohorts from unsupervised clustering algorithms optimized for various metrics. We investigate whether unsupervised clustering can reproduce ground truth classes in a labeled dataset, showing that success depends on feature selection and the chosen distance metric. Using toy datasets and fund categorization as real-world examples we demonstrate that accurately reproducing ground truth classes is challenging. We also highlight the limitations of standard clustering evaluation metrics in identifying the optimal number of clusters relative to the ground truth classes. We then show that if appropriate features are available in the dataset, and a proper distance metric is known (e.g., using a supervised Random Forest-based distance metric learning method), then an unsupervised clustering can indeed reproduce the ground truth classes as distinct clusters.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 8 of its 45 references that are in the map. First posted on arXiv 19 Aug 2024.

Builds on

8 of 45 references.

References · 8

  1. 2022ADBench: Anomaly Detection Benchmarkcited by 230
  2. 2022Geometry- and Accuracy-Preserving Random Forest Proximitiescited by 24
  3. 2019metric-learn: Metric Learning Algorithms in Pythoncited by 11
  4. 2023Quantifying Outlierness of Funds from their Categories using Supervised Similaritycited by 6
  5. 2020Machine Learning Fund Categorizationscited by 4
  6. 2024Open Set Recognition for Random Forestcited by 4
  7. 2024Quantile Regression using Random Forest Proximitiescited by 2
  8. 2024Case-based Explainability for Random Forest: Prototypes, Critics, Counter-factuals and Semi-factualscited by 1

Built on by

Cited by year

Citing papers per year: 2025: 2.

2 papers.