# High-Dimensional Distributed Sparse Classification with Scalable Communication-Efficient Global Updates

arXiv: https://arxiv.org/abs/2407.06346 · HTML: https://mlsanity.com/paper/2407.06346/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 45 references that are in the map.
- First posted on arXiv 8 Jul 2024.

## Builds on (most cited first, 14 of 14)

- [EMBER: An Open Dataset for Training Static PE Malware Machine Learning Models](https://mlsanity.com/paper/1804.04637/) (2018)
- [GIANT: Globally Improved Approximate Newton Method for Distributed Optimization](https://mlsanity.com/paper/1709.03528/) (2017)
- [Efficient Distributed Learning with Sparsity](https://mlsanity.com/paper/1605.07991/) (2016)
- [Communication-Efficient Accurate Statistical Estimation](https://mlsanity.com/paper/1906.04870/) (2019)
- [A General Framework for Auditing Differentially Private Machine Learning](https://mlsanity.com/paper/2210.08643/) (2022)
- [Distributed Coordinate Descent Method for Learning with Big Data](https://mlsanity.com/paper/1310.2059/) (2013)
- [On the Optimality of Averaging in Distributed Statistical Learning](https://mlsanity.com/paper/1407.2724/) (2014)
- [Scaling Up Differentially Private LASSO Regularized Logistic Regression via Faster Frank-Wolfe Iterations](https://mlsanity.com/paper/2310.19978/) (2023)
- [Communication-Efficient Distributed Optimization of Self-Concordant Empirical Loss](https://mlsanity.com/paper/1501.00263/) (2015)
- [Distributed Estimation, Information Loss and Exponential Families](https://mlsanity.com/paper/1410.2653/) (2014)
- [The Challenge of Differentially Private Screening Rules](https://mlsanity.com/paper/2303.10303/) (2023)
- [A Coreset Learning Reality Check](https://mlsanity.com/paper/2301.06163/) (2023)
- [SoK: A Review of Differentially Private Linear Models For High-Dimensional Data](https://mlsanity.com/paper/2404.01141/) (2024)
- [Distributed Coordinate Descent for L1-regularized Logistic Regression](https://mlsanity.com/paper/1411.6520/) (2014)

## Built on by (most cited first, 3 of 3)

- [Living off the Analyst: Harvesting Features from Yara Rules for Malware Detection](https://mlsanity.com/paper/2411.18516/) (2024)
- [Zipf-Gramming: Scaling Byte N-Grams Up to Production Sized Malware Corpora](https://mlsanity.com/paper/2511.13808/) (2025)
- [Simpler Methods Work Better for L1 Penalized Logistic Models and Large Datasets](https://mlsanity.com/paper/2609.23995/) (2026)

## Abstract (arXiv)

As the size of datasets used in statistical learning continues to grow, distributed training of models has attracted increasing attention. These methods partition the data and exploit parallelism to reduce memory and runtime, but suffer increasingly from communication costs as the data size or the number of iterations grows. Recent work on linear models has shown that a surrogate likelihood can be optimized locally to iteratively improve on an initial solution in a communication-efficient manner. However, existing versions of these methods experience multiple shortcomings as the data size becomes massive, including diverging updates and efficiently handling sparsity. In this work we develop solutions to these problems which enable us to learn a communication-efficient distributed logistic regression model even beyond millions of features. In our experiments we demonstrate a large improvement in accuracy over distributed algorithms with only a few distributed update steps needed, and similar or faster runtimes. Our code is available at urlhttps://github.com/FutureComputing4AI/ProxCSL.