The Fine-Grained Hardness of Sparse Linear Regression

Sparse linear regression is the well-studied inference problem where one is given a design matrix A ∈ ℝM× N and a response vector b ∈ ℝM, and the goal is to find a solution x ∈ ℝN which is k-sparse (that is, it has at most k non-zero coordinates) and minimizes the prediction error |A x - b|2. On the one hand, the problem is known to be NP-hard which tells us that no polynomial-time algorithm exists unless P = NP. On the other hand, the best known algorithms for the problem do a brute-force search among Nk possibilities. In this work, we show that there are no better-than-brute-force algorithms, assuming any one of a variety of popular conjectures including the weighted k-clique conjecture from the area of fine-grained complexity, or the hardness of the closest vector problem from the geometry of numbers. We also show the impossibility of better-than-brute-force algorithms when the prediction error is measured in other ℓp norms, assuming the strong exponential-time hypothesis.