# Parallel Selective Algorithms for Big Data Optimization

arXiv: https://arxiv.org/abs/1402.5521 · HTML: https://mlsanity.com/paper/1402.5521/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Feb 2014.

## Built on by (most cited first, 6 of 6)

- [A One-Sample Decentralized Proximal Algorithm for Non-Convex Stochastic Composite Optimization](https://mlsanity.com/paper/2302.09766/) (2023)
- [Multilinear Kernel Regression and Imputation via Manifold Learning](https://mlsanity.com/paper/2402.03648/) (2024)
- [Imputation of Time-varying Edge Flows in Graphs by Multilinear Kernel Regression and Manifold Learning](https://mlsanity.com/paper/2409.05135/) (2024)
- [Piecewise linear regression and classification](https://mlsanity.com/paper/2103.06189/) (2021)
- [A Randomised Subspace Gauss-Newton Method for Nonlinear Least-Squares](https://mlsanity.com/paper/2211.05727/) (2022)
- [Multi-Linear Kernel Regression and Imputation in Data Manifolds](https://mlsanity.com/paper/2304.03041/) (2023)

## Abstract (arXiv)

We propose a decomposition framework for the parallel optimization of the sum of a differentiable (possibly nonconvex) function and a (block) separable nonsmooth, convex one. The latter term is usually employed to enforce structure in the solution, typically sparsity. Our framework is very flexible and includes both fully parallel Jacobi schemes and Gauss- Seidel (i.e., sequential) ones, as well as virtually all possibilities "in between" with only a subset of variables updated at each iteration. Our theoretical convergence results improve on existing ones, and numerical results on LASSO, logistic regression, and some nonconvex quadratic problems show that the new method consistently outperforms existing algorithms.