# Core-Elements for Large-Scale Least Squares Estimation

arXiv: https://arxiv.org/abs/2206.10240 · HTML: https://mlsanity.com/paper/2206.10240/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Jun 2022.

## Built on by (most cited first, 3 of 3)

- [Knowledge Distillation and Dataset Distillation of Large Language Models: Emerging Trends, Challenges, and Future Directions](https://mlsanity.com/paper/2504.14772/) (2025)
- [An adaptive subsampling method for large-sample feature screening](https://mlsanity.com/paper/2509.16085/) (2025)
- [Core-elements Subsampling for Alternating Least Squares](https://mlsanity.com/paper/2509.18024/) (2025)

## Abstract (arXiv)

The coresets approach, also called subsampling or subset selection, aims to select a subsample as a surrogate for the observed sample and has found extensive applications in large-scale data analysis. Existing coresets methods construct the subsample using a subset of rows from the predictor matrix. Such methods can be significantly inefficient when the predictor matrix is sparse or numerically sparse. To overcome this limitation, we develop a novel element-wise subset selection approach, called core-elements, for large-scale least squares estimation. We provide a deterministic algorithm to construct the core-elements estimator, only requiring an O(nnz(X)+rp2) computational cost, where X is an n× p predictor matrix, r is the number of elements selected from each column of X, and nnz(·) denotes the number of non-zero elements. Theoretically, we show that the proposed estimator is unbiased and approximately minimizes an upper bound of the estimation variance. We also provide an approximation guarantee by deriving a coresets-like finite sample bound for the proposed estimator. To handle potential outliers in the data, we further combine core-elements with the median-of-means procedure, resulting in an efficient and robust estimator with theoretical consistency guarantees. Numerical studies on various synthetic and real-world datasets demonstrate the proposed method's superior performance compared to mainstream competitors.