In-Database Regression in Input Sparsity Time

Sketching is a powerful dimensionality reduction technique for accelerating algorithms for data analysis. A crucial step in sketching methods is to compute a subspace embedding (SE) for a large matrix A ∈ ℝN × d. SE's are the primary tool for obtaining extremely efficient solutions for many linear-algebraic tasks, such as least squares regression and low rank approximation. Computing an SE often requires an explicit representation of A and running time proportional to the size of A. However, if A= T1 Join T2 Join dots Join Tm is the result of a database join query on several smaller tables Ti ∈ ℝni × di, then this running time can be prohibitive, as A itself can have as many as O(n1 n2 cdots nm) rows. In this work, we design subspace embeddings for database joins which can be computed significantly faster than computing the join. For the case of a two table join A = T1 Join T2 we give input-sparsity algorithms for computing subspace embeddings, with running time bounded by the number of non-zero entries in T1,T2. This results in input-sparsity time algorithms for high accuracy regression, significantly improving upon the running time of prior FAQ-based methods for regression. We extend our results to arbitrary joins for the ridge regression problem, also considerably improving the running time of prior methods. Empirically, we apply our method to real datasets and show that it is significantly faster than existing algorithms.