# Distributed Sparse Linear Regression under Communication Constraints

arXiv: https://arxiv.org/abs/2301.04022 · HTML: https://mlsanity.com/paper/2301.04022/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 19 of its 53 references that are in the map.
- First posted on arXiv 9 Jan 2023.

## Builds on (most cited first, 19 of 19)

- [Advances and Open Problems in Federated Learning](https://mlsanity.com/paper/1912.04977/) (2019)
- [Towards Federated Learning at Scale: System Design](https://mlsanity.com/paper/1902.01046/) (2019)
- [Confidence Intervals and Hypothesis Testing for High-Dimensional Regression](https://mlsanity.com/paper/1306.3171/) (2013)
- [Communication Lower Bounds for Statistical Estimation Problems via a Distributed Data Processing Inequality](https://mlsanity.com/paper/1506.07216/) (2015)
- [De-biasing the Lasso: Optimal Sample Size for Gaussian Designs](https://mlsanity.com/paper/1508.02757/) (2015)
- [Distributed linear regression by averaging](https://mlsanity.com/paper/1810.00412/) (2018)
- [Hypothesis Testing in High-Dimensional Regression under the Gaussian Random Design Model: Asymptotic Theory](https://mlsanity.com/paper/1301.4240/) (2013)
- [On the Optimality of Averaging in Distributed Statistical Learning](https://mlsanity.com/paper/1407.2724/) (2014)
- [Distributed High-dimensional Regression Under a Quantile Loss Function](https://mlsanity.com/paper/1906.05741/) (2019)
- [Adaptive distributed methods under communication constraints](https://mlsanity.com/paper/1804.00864/) (2018)
- [Distributed Learning with Sublinear Communication](https://mlsanity.com/paper/1902.11259/) (2019)
- [Least Squares Approximation for a Distributed System](https://mlsanity.com/paper/1908.04904/) (2019)
- [DUAL-LOCO: Distributing Statistical Estimation Using Random Projections](https://mlsanity.com/paper/1506.02554/) (2015)
- [Integrative High Dimensional Multiple Testing with Heterogeneity under Data Sharing Constraints](https://mlsanity.com/paper/2004.00816/) (2020)
- [LOCO: Distributing Ridge Regression with Random Projections](https://mlsanity.com/paper/1406.3469/) (2014)
- [Distributed Sparse Normal Means Estimation with Sublinear Communication](https://mlsanity.com/paper/2102.03060/) (2021)
- [Debiased distributed learning for sparse partial linear models in high dimensions](https://mlsanity.com/paper/1708.05487/) (2017)
- [Distributed Sparse Feature Selection in Communication-Restricted Networks](https://mlsanity.com/paper/2111.02802/) (2021)
- [Distributed Sparse Regression via Penalization](https://mlsanity.com/paper/2111.06530/) (2021)

## Abstract (arXiv)

In multiple domains, statistical tasks are performed in distributed settings, with data split among several end machines that are connected to a fusion center. In various applications, the end machines have limited bandwidth and power, and thus a tight communication budget. In this work we focus on distributed learning of a sparse linear regression model, under severe communication constraints. We propose several two round distributed schemes, whose communication per machine is sublinear in the data dimension. In our schemes, individual machines compute debiased lasso estimators, but send to the fusion center only very few values. On the theoretical front, we analyze one of these schemes and prove that with high probability it achieves exact support recovery at low signal to noise ratios, where individual machines fail to recover the support. We show in simulations that our scheme works as well as, and in some cases better, than more communication intensive approaches.