# Ridge Regression with Frequent Directions: Statistical and Optimization Perspectives

arXiv: https://arxiv.org/abs/2011.03607 · HTML: https://mlsanity.com/paper/2011.03607/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Nov 2020.

## Built on by (most cited first, 1 of 1)

- [Optimal Matrix Sketching over Sliding Windows](https://mlsanity.com/paper/2405.07792/) (2024)

## Abstract (arXiv)

Despite its impressive theory & practical performance, Frequent Directions (acrshortfd) has not been widely adopted for large-scale regression tasks. Prior work has shown randomized sketches (i) perform worse in estimating the covariance matrix of the data than acrshortfd; (ii) incur high error when estimating the bias and/or variance on sketched ridge regression. We give the first constant factor relative error bounds on the bias & variance for sketched ridge regression using acrshortfd. We complement these statistical results by showing that acrshortfd can be used in the optimization setting through an iterative scheme which yields high-accuracy solutions. This improves on randomized approaches which need to compromise the need for a new sketch every iteration with speed of convergence. In both settings, we also show using Robust Frequent Directions further enhances performance.