# Distributed Semi-Supervised Sparse Statistical Inference

arXiv: https://arxiv.org/abs/2306.10395 · HTML: https://mlsanity.com/paper/2306.10395/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 73 references that are in the map.
- First posted on arXiv 17 Jun 2023.

## Builds on (most cited first, 29 of 29)

- [Local SGD Converges Fast and Communicates Little](https://mlsanity.com/paper/1805.09767/) (2018)
- [Communication Efficient Distributed Optimization using an Approximate Newton-type Method](https://mlsanity.com/paper/1312.7853/) (2013)
- [Confidence Intervals and Hypothesis Testing for High-Dimensional Regression](https://mlsanity.com/paper/1306.3171/) (2013)
- [CoCoA: A General Framework for Communication-Efficient Distributed Optimization](https://mlsanity.com/paper/1611.02189/) (2016)
- [Divide and Conquer Kernel Ridge Regression: A Distributed Algorithm with Minimax Optimal Rates](https://mlsanity.com/paper/1305.5029/) (2013)
- [Communication Complexity of Distributed Convex Learning and Optimization](https://mlsanity.com/paper/1506.01900/) (2015)
- [On Iterative Hard Thresholding Methods for High-dimensional M-Estimation](https://mlsanity.com/paper/1410.5137/) (2014)
- [Distributed learning with regularized least squares](https://mlsanity.com/paper/1608.03339/) (2016)
- [Moving Beyond Sub-Gaussianity in High-Dimensional Statistics: Applications in Covariance Estimation and Linear Regression](https://mlsanity.com/paper/1804.02605/) (2018)
- [Efficient Distributed Learning with Sparsity](https://mlsanity.com/paper/1605.07991/) (2016)
- [Semi-supervised Inference: General Theory and Estimation of Means](https://mlsanity.com/paper/1606.07268/) (2016)
- [Communication-Efficient Accurate Statistical Estimation](https://mlsanity.com/paper/1906.04870/) (2019)
- [Efficient and Adaptive Linear Regression in Semi-Supervised Settings](https://mlsanity.com/paper/1701.04889/) (2017)
- [Quantile Regression Under Memory Constraint](https://mlsanity.com/paper/1810.08264/) (2018)
- [Semi-supervised Inference for Explained Variance in High-dimensional Linear Regression and Its Applications](https://mlsanity.com/paper/1806.06179/) (2018)
- [Distributed Kernel Ridge Regression with Communications](https://mlsanity.com/paper/2003.12210/) (2020)
- [Distributed High-dimensional Regression Under a Quantile Loss Function](https://mlsanity.com/paper/1906.05741/) (2019)
- [A Distributed One-Step Estimator](https://mlsanity.com/paper/1511.01443/) (2015)
- [Distributed inference for quantile regression processes](https://mlsanity.com/paper/1701.06088/) (2017)
- [Global and Simultaneous Hypothesis Testing for High-Dimensional Logistic Regression Models](https://mlsanity.com/paper/1805.06970/) (2018)
- [Optimal and Safe Estimation for High-Dimensional Semi-Supervised Learning](https://mlsanity.com/paper/2011.14185/) (2020)
- [Semi-Supervised Approaches to Efficient Evaluation of Model Prediction Performance](https://mlsanity.com/paper/1711.05663/) (2017)
- [Integrative High Dimensional Multiple Testing with Heterogeneity under Data Sharing Constraints](https://mlsanity.com/paper/2004.00816/) (2020)
- [A General Framework for Robust Testing and Confidence Regions in High-Dimensional Quantile Regression](https://mlsanity.com/paper/1412.8724/) (2014)
- [Learning Theory of Distributed Regression with Bias Corrected Regularization Kernel Network](https://mlsanity.com/paper/1708.01960/) (2017)
- [Inference for the Case Probability in High-dimensional Logistic Regression](https://mlsanity.com/paper/2012.07133/) (2020)
- [An efficient ADMM algorithm for high dimensional precision matrix estimation via penalized quadratic loss](https://mlsanity.com/paper/1811.04545/) (2018)
- [A unified precision matrix estimation framework via sparse column-wise inverse operator under weak sparsity](https://mlsanity.com/paper/2107.02999/) (2021)
- [Statistical Inference for Genetic Relatedness Based on High-Dimensional Logistic Regression](https://mlsanity.com/paper/2202.10007/) (2022)

## Built on by (most cited first, 1 of 1)

- [A Selective Review on Statistical Methods for Massive Data Computation: Distributed Computing, Subsampling, and Minibatch Techniques](https://mlsanity.com/paper/2403.11163/) (2024)

## Abstract (arXiv)

The debiased estimator is a crucial tool in statistical inference for high-dimensional model parameters. However, constructing such an estimator involves estimating the high-dimensional inverse Hessian matrix, incurring significant computational costs. This challenge becomes particularly acute in distributed setups, where traditional methods necessitate computing a debiased estimator on every machine. This becomes unwieldy, especially with a large number of machines. In this paper, we delve into semi-supervised sparse statistical inference in a distributed setup. An efficient multi-round distributed debiased estimator, which integrates both labeled and unlabelled data, is developed. We will show that the additional unlabeled data helps to improve the statistical rate of each round of iteration. Our approach offers tailored debiasing methods for M-estimation and generalized linear models according to the specific form of the loss function. Our method also applies to a non-smooth loss like absolute deviation loss. Furthermore, our algorithm is computationally efficient since it requires only one estimation of a high-dimensional inverse covariance matrix. We demonstrate the effectiveness of our method by presenting simulation studies and real data applications that highlight the benefits of incorporating unlabeled data.