# Estimating the Optimal Number of Clusters in Categorical Data Clustering by Silhouette Coefficient

arXiv: https://arxiv.org/abs/2501.15542 · HTML: https://mlsanity.com/paper/2501.15542/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 0 of its 21 references that are in the map.
- First posted on arXiv 26 Jan 2025.

## Built on by (most cited first, 6 of 6)

- [Decouple and Track: Benchmarking and Improving Video Diffusion Transformers for Motion Transfer](https://mlsanity.com/paper/2503.17350/) (2025)
- [HiSAC: Hierarchical Sparse Activation Compression for Ultra-long Sequence Modeling in Recommenders](https://mlsanity.com/paper/2602.21009/) (2026)
- [An Explainable Neural Radiomic Sequence Model with Spatiotemporal Continuity for Quantifying 4DCT-based Pulmonary Ventilation](https://mlsanity.com/paper/2503.23898/) (2025)
- [An upper bound on the silhouette evaluation metric for clustering](https://mlsanity.com/paper/2509.08625/) (2025)
- [An improved clustering-based multi-swarm PSO using local diversification and topology information](https://mlsanity.com/paper/2511.17571/) (2025)
- [Scalable and Distributed Silhouette Approximation](https://mlsanity.com/paper/2607.01993/) (2026)

## Abstract (arXiv)

The problem of estimating the number of clusters (say k) is one of the major challenges for the partitional clustering. This paper proposes an algorithm named k-SCC to estimate the optimal k in categorical data clustering. For the clustering step, the algorithm uses the kernel density estimation approach to define cluster centers. In addition, it uses an information-theoretic based dissimilarity to measure the distance between centers and objects in each cluster. The silhouette analysis based approach is then used to evaluate the quality of different clustering obtained in the former step to choose the best k. Comparative experiments were conducted on both synthetic and real datasets to compare the performance of k-SCC with three other algorithms. Experimental results show that k-SCC outperforms the compared algorithms in determining the number of clusters for each dataset.