Column Subset Selection and Nystr"om Approximation via Continuous Optimization

We propose a continuous optimization algorithm for the Column Subset Selection Problem (CSSP) and Nystr"om approximation. The CSSP and Nystr"om method construct low-rank approximations of matrices based on a predetermined subset of columns. It is well known that choosing the best column subset of size k is a difficult combinatorial problem. In this work, we show how one can approximate the optimal solution by defining a penalized continuous loss function which is minimized via stochastic gradient descent. We show that the gradients of this loss function can be estimated efficiently using matrix-vector products with a data matrix X in the case of the CSSP or a kernel matrix K in the case of the Nystr"om approximation. We provide numerical results for a number of real datasets showing that this continuous optimization is competitive against existing methods.