A Fast Algorithm for Clustering High Dimensional Feature Vectors

We propose an algorithm for clustering high dimensional data. If P features for N objects are represented in an N× P matrix bf X, where Nll P, the method is based on exploiting the cluster-dependent structure of the N× N matrix bf XXT. Computational burden thus depends primarily on N, the number of objects to be clustered, rather than P, the number of features that are measured. This makes the method particularly useful in high dimensional settings, where it is substantially faster than a number of other popular clustering algorithms. Aside from an upper bound on the number of potential clusters, the method is independent of tuning parameters. When compared to 16 other clustering algorithms on 32 genomic datasets with gold standards, we show that it provides the most accurate cluster configuration more than twice as often than its closest competitors. We illustrate the method on data taken from highly cited genomic studies.