# Optimal Subsampling for Large Sample Logistic Regression

arXiv: https://arxiv.org/abs/1702.01166 · HTML: https://mlsanity.com/paper/1702.01166/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Feb 2017.

## Built on by (most cited first, 29 of 29)

- [Coresets for Classification -- Simplified and Strengthened](https://mlsanity.com/paper/2106.04254/) (2021)
- [Towards a statistical theory of data selection under weak supervision](https://mlsanity.com/paper/2309.14563/) (2023)
- [Gaussian Switch Sampling: A Second Order Approach to Active Learning](https://mlsanity.com/paper/2302.12018/) (2023)
- [A Coreset Learning Reality Check](https://mlsanity.com/paper/2301.06163/) (2023)
- [Sketchy Moment Matching: Toward Fast and Provable Data Selection for Finetuning](https://mlsanity.com/paper/2407.06120/) (2024)
- [One Backward from Ten Forward, Subsampling for Large-Scale Deep Learning](https://mlsanity.com/paper/2104.13114/) (2021)
- [Nonuniform Negative Sampling and Log Odds Correction with Rare Events Data](https://mlsanity.com/paper/2110.13048/) (2021)
- [An optimal transport approach for selecting a representative subsample with application in efficient kernel density estimation](https://mlsanity.com/paper/2206.01182/) (2022)
- [Optimal Sample Selection Through Uncertainty Estimation and Its Application in Deep Learning](https://mlsanity.com/paper/2309.02476/) (2023)
- [Model-free Subsampling Method Based on Uniform Designs](https://mlsanity.com/paper/2209.03617/) (2022)
- [Train on Validation (ToV): Fast data selection with applications to fine-tuning](https://mlsanity.com/paper/2510.00386/) (2025)
- [General bounds on the quality of Bayesian coresets](https://mlsanity.com/paper/2405.11780/) (2024)
- [Fundamental Bias in Inverting Random Sampling Matrices with Application to Sub-sampled Newton](https://mlsanity.com/paper/2502.13583/) (2025)
- [On the Subbagging Estimation for Massive Data](https://mlsanity.com/paper/2103.00631/) (2021)
- [Deep Neural Network Based Accelerated Failure Time Models using Rank Loss](https://mlsanity.com/paper/2206.05974/) (2022)
- [A sub-sampling algorithm preventing outliers](https://mlsanity.com/paper/2208.06218/) (2022)
- [Optimal Sampling Designs for Multi-dimensional Streaming Time Series with Application to Power Grid Sensor Data](https://mlsanity.com/paper/2303.08242/) (2023)
- [A Provably Accurate Randomized Sampling Algorithm for Logistic Regression](https://mlsanity.com/paper/2402.16326/) (2024)
- [A Selective Review on Statistical Methods for Massive Data Computation: Distributed Computing, Subsampling, and Minibatch Techniques](https://mlsanity.com/paper/2403.11163/) (2024)
- [A model-free subdata selection method for classification](https://mlsanity.com/paper/2404.19127/) (2024)
- [Data Compression using Rank-1 Lattices for Parameter Estimation in Machine Learning](https://mlsanity.com/paper/2409.13453/) (2024)
- [Optimal Sampling for Generalized Linear Model under Measurement Constraint with Surrogate Variables](https://mlsanity.com/paper/2501.00972/) (2025)
- [Sublinear Algorithms for Wasserstein and Total Variation Distances: Applications to Fairness and Privacy Auditing](https://mlsanity.com/paper/2503.07775/) (2025)
- [Core-elements Subsampling for Alternating Least Squares](https://mlsanity.com/paper/2509.18024/) (2025)
- [Learning U-Statistics with Active Inference](https://mlsanity.com/paper/2605.11638/) (2026)
- [Knowledge Cascade: Reverse Knowledge Distillation on Nonparametric Multivariate Functional Estimation](https://mlsanity.com/paper/2606.25927/) (2026)
- [Influence Diagnostics in High-dimensional M-estimation: Precise Asymptotics](https://mlsanity.com/paper/2607.09250/) (2026)
- [Scale-invariant Optimal Sampling for Rare-events Data with Sparse Models](https://mlsanity.com/paper/2608.22597/) (2026)
- [Subdata Selection for Measurement-Constrained Linear Regression under Model Misspecification](https://mlsanity.com/paper/2610.19351/) (2026)

## Abstract (arXiv)

For massive data, the family of subsampling algorithms is popular to downsize the data volume and reduce computational burden. Existing studies focus on approximating the ordinary least squares estimate in linear regression, where statistical leverage scores are often used to define subsampling probabilities. In this paper, we propose fast subsampling algorithms to efficiently approximate the maximum likelihood estimate in logistic regression. We first establish consistency and asymptotic normality of the estimator from a general subsampling algorithm, and then derive optimal subsampling probabilities that minimize the asymptotic mean squared error of the resultant estimator. An alternative minimization criterion is also proposed to further reduce the computational cost. The optimal subsampling probabilities depend on the full data estimate, so we develop a two-step algorithm to approximate the optimal subsampling procedure. This algorithm is computationally efficient and has a significant reduction in computing time compared to the full data approach. Consistency and asymptotic normality of the estimator from a two-step algorithm are also established. Synthetic and real data sets are used to evaluate the practical performance of the proposed method.