# Smoothing ADMM for Sparse-Penalized Quantile Regression with Non-Convex Penalties

arXiv: https://arxiv.org/abs/2309.03094 · HTML: https://mlsanity.com/paper/2309.03094/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 45 references that are in the map.
- First posted on arXiv 4 Sep 2023.

## Builds on (most cited first, 11 of 11)

- [Stochastic model-based minimization of weakly convex functions](https://mlsanity.com/paper/1803.06523/) (2018)
- [Global Convergence of ADMM in Nonconvex Nonsmooth Optimization](https://mlsanity.com/paper/1511.06324/) (2015)
- [Lower complexity bounds of first-order methods for convex-concave bilinear saddle-point problems](https://mlsanity.com/paper/1808.02901/) (2018)
- [Convergence Analysis of Alternating Direction Method of Multipliers for a Family of Nonconvex Problems](https://mlsanity.com/paper/1410.1390/) (2014)
- [Adaptive piecewise polynomial estimation via trend filtering](https://mlsanity.com/paper/1304.2986/) (2013)
- [Subgradient methods for sharp weakly convex functions](https://mlsanity.com/paper/1803.02461/) (2018)
- [Douglas-Rachford splitting and ADMM for nonconvex optimization: tight convergence results](https://mlsanity.com/paper/1709.05747/) (2017)
- [Distributed Stochastic Gradient Descent: Nonconvexity, Nonsmoothness, and Convergence to Local Minima](https://mlsanity.com/paper/2003.02818/) (2020)
- [Vector-Valued Graph Trend Filtering with Non-Convex Penalties](https://mlsanity.com/paper/1905.12692/) (2019)
- [Moreau Envelope Augmented Lagrangian Method for Nonconvex Optimization with Linear Constraints](https://mlsanity.com/paper/2101.08519/) (2021)
- [Smoothing Gradient Tracking for Decentralized Optimization over the Stiefel Manifold with Non-smooth Regularizers](https://mlsanity.com/paper/2303.15882/) (2023)

## Built on by (most cited first, 4 of 4)

- [Decentralized Smoothing ADMM for Quantile Regression with Non-Convex Sparse Penalties](https://mlsanity.com/paper/2408.01307/) (2024)
- [Federated Smoothing Proximal Gradient for Quantile Regression with Non-Convex Penalties](https://mlsanity.com/paper/2408.05640/) (2024)
- [Smoothing ADMM for Non-convex and Non-smooth Hierarchical Federated Learning](https://mlsanity.com/paper/2503.08869/) (2025)
- [Federated Smoothing ADMM for Localization](https://mlsanity.com/paper/2503.09497/) (2025)

## Abstract (arXiv)

This paper investigates quantile regression in the presence of non-convex and non-smooth sparse penalties, such as the minimax concave penalty (MCP) and smoothly clipped absolute deviation (SCAD). The non-smooth and non-convex nature of these problems often leads to convergence difficulties for many algorithms. While iterative techniques like coordinate descent and local linear approximation can facilitate convergence, the process is often slow. This sluggish pace is primarily due to the need to run these approximation techniques until full convergence at each step, a requirement we term as a secondary convergence iteration. To accelerate the convergence speed, we employ the alternating direction method of multipliers (ADMM) and introduce a novel single-loop smoothing ADMM algorithm with an increasing penalty parameter, named SIAD, specifically tailored for sparse-penalized quantile regression. We first delve into the convergence properties of the proposed SIAD algorithm and establish the necessary conditions for convergence. Theoretically, we confirm a convergence rate of obig(k^-frac14big) for the sub-gradient bound of augmented Lagrangian. Subsequently, we provide numerical results to showcase the effectiveness of the SIAD algorithm. Our findings highlight that the SIAD method outperforms existing approaches, providing a faster and more stable solution for sparse-penalized quantile regression.