# Continuation Path with Linear Convergence Rate

arXiv: https://arxiv.org/abs/2112.05104 · HTML: https://mlsanity.com/paper/2112.05104/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 25 references that are in the map.
- First posted on arXiv 9 Dec 2021.

## Builds on (most cited first, 7 of 7)

- [Gap Safe screening rules for sparsity enforcing penalties](https://mlsanity.com/paper/1611.05780/) (2016)
- [Mind the duality gap: safer rules for the Lasso](https://mlsanity.com/paper/1505.03410/) (2015)
- [Celer: a Fast Solver for the Lasso with Dual Extrapolation](https://mlsanity.com/paper/1802.07481/) (2018)
- [Safe Grid Search with Optimal Complexity](https://mlsanity.com/paper/1810.05471/) (2018)
- [Screening Rules and its Complexity for Active Set Identification](https://mlsanity.com/paper/2009.02709/) (2020)
- [Pathwise Coordinate Optimization for Sparse Learning: Algorithm and Theory](https://mlsanity.com/paper/1412.7477/) (2014)
- [The Hessian Screening Rule](https://mlsanity.com/paper/2104.13026/) (2021)

## Built on by (most cited first, 3 of 3)

- [Beyond L1: Faster and Better Sparse Models with skglm](https://mlsanity.com/paper/2204.07826/) (2022)
- [Conformalization of Sparse Generalized Linear Models](https://mlsanity.com/paper/2307.05109/) (2023)
- [Beyond Discretization: Learning the Optimal Solution Path](https://mlsanity.com/paper/2410.14885/) (2024)

## Abstract (arXiv)

Path-following algorithms are frequently used in composite optimization problems where a series of subproblems, with varying regularization hyperparameters, are solved sequentially. By reusing the previous solutions as initialization, better convergence speeds have been observed numerically. This makes it a rather useful heuristic to speed up the execution of optimization algorithms in machine learning. We present a primal dual analysis of the path-following algorithm and explore how to design its hyperparameters as well as determining how accurately each subproblem should be solved to guarantee a linear convergence rate on a target problem. Furthermore, considering optimization with a sparsity-inducing penalty, we analyze the change of the active sets with respect to the regularization parameter. The latter can then be adaptively calibrated to finely determine the number of features that will be selected along the solution path. This leads to simple heuristics for calibrating hyperparameters of active set approaches to reduce their complexity and improve their execution time.