# High-dimensional online learning via asynchronous decomposition: Non-divergent results, dynamic regularization, and beyond

arXiv: https://arxiv.org/abs/2603.20696 · HTML: https://mlsanity.com/paper/2603.20696/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 10 of its 47 references that are in the map.
- First posted on arXiv 21 Mar 2026.

## Builds on (most cited first, 10 of 10)

- [Statistical Inference for Model Parameters in Stochastic Gradient Descent](https://mlsanity.com/paper/1610.08637/) (2016)
- [On Iterative Hard Thresholding Methods for High-dimensional M-Estimation](https://mlsanity.com/paper/1410.5137/) (2014)
- [Near-optimal inference in adaptive linear regression](https://mlsanity.com/paper/2107.02266/) (2021)
- [High-Dimensional Quantile Regression: Convolution Smoothing and Concave Regularization](https://mlsanity.com/paper/2109.05640/) (2021)
- [Model selection and minimax estimation in generalized linear models](https://mlsanity.com/paper/1409.8491/) (2014)
- [Sparse recovery by reduced variance stochastic approximation](https://mlsanity.com/paper/2006.06365/) (2020)
- [Statistical Inference for Streamed Longitudinal Data](https://mlsanity.com/paper/2208.02890/) (2022)
- [Inference on High-dimensional Single-index Models with Streaming Data](https://mlsanity.com/paper/2210.00937/) (2022)
- [Slow Kill for Big Data Learning](https://mlsanity.com/paper/2305.01726/) (2023)
- [Adaptive debiased SGD in high-dimensional GLMs with streaming data](https://mlsanity.com/paper/2405.18284/) (2024)

## Abstract (arXiv)

Existing high-dimensional online learning methods often face the challenge that their error bounds, or per-batch sample sizes, diverge as the number of data batches increases. To address this issue, we propose an asynchronous decomposition framework that leverages summary statistics to construct a surrogate score function for current-batch learning. This framework is implemented via a dynamic-regularized iterative hard thresholding algorithm, providing a computationally and memory-efficient solution for sparse online optimization. We provide a unified theoretical analysis that accounts for both the streaming computational error and statistical accuracy, establishing that our estimator maintains non-divergent error bounds and ℓ0 sparsity across all batches. Furthermore, the proposed estimator adaptively achieves additional gains as batches accumulate, attaining the oracle accuracy as if the entire historical dataset were accessible and the true support were known. These theoretical properties are further illustrated through an example of the generalized linear model.