# A GPU-Accelerated Bi-linear ADMM Algorithm for Distributed Sparse Machine Learning

arXiv: https://arxiv.org/abs/2405.16267 · HTML: https://mlsanity.com/paper/2405.16267/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 25 references that are in the map.
- First posted on arXiv 25 May 2024.

## Builds on (most cited first, 11 of 11)

- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [A Survey on Distributed Machine Learning](https://mlsanity.com/paper/1912.09789/) (2019)
- [Best Subset Selection via a Modern Optimization Lens](https://mlsanity.com/paper/1507.03133/) (2015)
- [From Distributed Machine Learning to Federated Learning: A Survey](https://mlsanity.com/paper/2104.14362/) (2021)
- [Sparse High-Dimensional Regression: Exact Scalable Algorithms and Phase Transitions](https://mlsanity.com/paper/1709.10029/) (2017)
- [Sparse Classification: a scalable discrete optimization perspective](https://mlsanity.com/paper/1710.01352/) (2017)
- [Cardinality Minimization, Constraints, and Regularization: A Survey](https://mlsanity.com/paper/2106.09606/) (2021)
- [Newton-ADMM: A Distributed GPU-Accelerated Optimizer for Multiclass Classification Problems](https://mlsanity.com/paper/1807.07132/) (2018)
- [Triggered Gradient Tracking for Asynchronous Distributed Optimization](https://mlsanity.com/paper/2203.02210/) (2022)
- [Distributed Primal Outer Approximation Algorithm for Sparse Convex Programming with Separable Structures](https://mlsanity.com/paper/2210.06913/) (2022)
- [Sparse Convex Optimization Toolkit: A Mixed-Integer Framework](https://mlsanity.com/paper/2210.16896/) (2022)

## Abstract (arXiv)

This paper introduces the Bi-linear consensus Alternating Direction Method of Multipliers (Bi-cADMM), aimed at solving large-scale regularized Sparse Machine Learning (SML) problems defined over a network of computational nodes. Mathematically, these are stated as minimization problems with convex local loss functions over a global decision vector, subject to an explicit ℓ0 norm constraint to enforce the desired sparsity. The considered SML problem generalizes different sparse regression and classification models, such as sparse linear and logistic regression, sparse softmax regression, and sparse support vector machines. Bi-cADMM leverages a bi-linear consensus reformulation of the original non-convex SML problem and a hierarchical decomposition strategy that divides the problem into smaller sub-problems amenable to parallel computing. In Bi-cADMM, this decomposition strategy is based on a two-phase approach. Initially, it performs a sample decomposition of the data and distributes local datasets across computational nodes. Subsequently, a delayed feature decomposition of the data is conducted on Graphics Processing Units (GPUs) available to each node. This methodology allows Bi-cADMM to undertake computationally intensive data-centric computations on GPUs, while CPUs handle more cost-effective computations. The proposed algorithm is implemented within an open-source Python package called Parallel Sparse Fitting Toolbox (PsFiT), which is publicly available. Finally, computational experiments demonstrate the efficiency and scalability of our algorithm through numerical benchmarks across various SML problems featuring distributed datasets.