# Distributed Sparse Feature Selection in Communication-Restricted Networks

arXiv: https://arxiv.org/abs/2111.02802 · HTML: https://mlsanity.com/paper/2111.02802/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 45 references that are in the map.
- First posted on arXiv 2 Nov 2021.

## Builds on (most cited first, 12 of 12)

- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Controlling the false discovery rate via knockoffs](https://mlsanity.com/paper/1404.5609/) (2014)
- [CoCoA: A General Framework for Communication-Efficient Distributed Optimization](https://mlsanity.com/paper/1611.02189/) (2016)
- [Efficient Distributed Learning with Sparsity](https://mlsanity.com/paper/1605.07991/) (2016)
- [De-biasing the Lasso: Optimal Sample Size for Gaussian Designs](https://mlsanity.com/paper/1508.02757/) (2015)
- [A General Analysis of the Convergence of ADMM](https://mlsanity.com/paper/1502.02009/) (2015)
- [Hypothesis Testing in High-Dimensional Regression under the Gaussian Random Design Model: Asymptotic Theory](https://mlsanity.com/paper/1301.4240/) (2013)
- [Distributed High-dimensional Regression Under a Quantile Loss Function](https://mlsanity.com/paper/1906.05741/) (2019)
- [False Discovery Rate Control via Debiased Lasso](https://mlsanity.com/paper/1803.04464/) (2018)
- [Locally Convex Sparse Learning over Networks](https://mlsanity.com/paper/1804.00130/) (2018)
- [BELIEF: A distance-based redundancy-proof feature selection method for Big Data](https://mlsanity.com/paper/1804.05774/) (2018)
- [ADAGES: adaptive aggregation with stability for distributed feature selection](https://mlsanity.com/paper/2007.10776/) (2020)

## Built on by (most cited first, 2 of 2)

- [Recovery Guarantees for Distributed-OMP](https://mlsanity.com/paper/2209.07230/) (2022)
- [Distributed Sparse Linear Regression under Communication Constraints](https://mlsanity.com/paper/2301.04022/) (2023)

## Abstract (arXiv)

This paper aims to propose and theoretically analyze a new distributed scheme for sparse linear regression and feature selection. The primary goal is to learn the few causal features of a high-dimensional dataset based on noisy observations from an unknown sparse linear model. However, the presumed training set which includes n data samples in ℝp is already distributed over a large network with N clients connected through extremely low-bandwidth links. Also, we consider the asymptotic configuration of 1ll Nll nll p. In order to infer the causal dimensions from the whole dataset, we propose a simple, yet effective method for information sharing in the network. In this regard, we theoretically show that the true causal features can be reliably recovered with negligible bandwidth usage of Oleft(Nlog pright) across the network. This yields a significantly lower communication cost in comparison with the trivial case of transmitting all the samples to a single node (centralized scenario), which requires Oleft(npright) transmissions. Even more sophisticated schemes such as ADMM still have a communication complexity of Oleft(Npright). Surprisingly, our sample complexity bound is proved to be the same (up to a constant factor) as the optimal centralized approach for a fixed performance measure in each node, while that of a na"ive decentralized technique grows linearly with N. Theoretical guarantees in this paper are based on the recent analytic framework of debiased LASSO in Javanmard et al. (2019), and are supported by several computer experiments performed on both synthetic and real-world datasets.