# Orthogonal Subsampling for Big Data Linear Regression

arXiv: https://arxiv.org/abs/2105.14647 · HTML: https://mlsanity.com/paper/2105.14647/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 May 2021.

## Built on by (most cited first, 6 of 6)

- [Optimal Sampling Designs for Multi-dimensional Streaming Time Series with Application to Power Grid Sensor Data](https://mlsanity.com/paper/2303.08242/) (2023)
- [An Overview of Healthcare Data Analytics With Applications to the COVID-19 Pandemic](https://mlsanity.com/paper/2111.14623/) (2021)
- [Leveraging Large Language Models for Rare Disease Named Entity Recognition](https://mlsanity.com/paper/2508.09323/) (2025)
- [Core-elements Subsampling for Alternating Least Squares](https://mlsanity.com/paper/2509.18024/) (2025)
- [Nearly Optimal Subdata Selection](https://mlsanity.com/paper/2604.23930/) (2026)
- [Subdata Selection for Measurement-Constrained Linear Regression under Model Misspecification](https://mlsanity.com/paper/2610.19351/) (2026)

## Abstract (arXiv)

The dramatic growth of big datasets presents a new challenge to data storage and analysis. Data reduction, or subsampling, that extracts useful information from datasets is a crucial step in big data analysis. We propose an orthogonal subsampling (OSS) approach for big data with a focus on linear regression models. The approach is inspired by the fact that an orthogonal array of two levels provides the best experimental design for linear regression models in the sense that it minimizes the average variance of the estimated parameters and provides the best predictions. The merits of OSS are three-fold: (i) it is easy to implement and fast; (ii) it is suitable for distributed parallel computing and ensures the subsamples selected in different batches have no common data points; and (iii) it outperforms existing methods in minimizing the mean squared errors of the estimated parameters and maximizing the efficiencies of the selected subsamples. Theoretical results and extensive numerical results show that the OSS approach is superior to existing subsampling approaches. It is also more robust to the presence of interactions among covariates and, when they do exist, OSS provides more precise estimates of the interaction effects than existing methods. The advantages of OSS are also illustrated through analysis of real data.