# WEEP: A Differentiable Nonconvex Sparse Regularizer via Weakly-Convex Envelope

arXiv: https://arxiv.org/abs/2507.20447 · HTML: https://mlsanity.com/paper/2507.20447/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 67 references that are in the map.
- First posted on arXiv 28 Jul 2025.

## Builds on (most cited first, 21 of 21)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [An overview of gradient descent optimization algorithms](https://mlsanity.com/paper/1609.04747/) (2016)
- [Learning Sparse Neural Networks through L0 Regularization](https://mlsanity.com/paper/1712.01312/) (2017)
- [Real-world Noisy Image Denoising: A New Benchmark](https://mlsanity.com/paper/1804.02603/) (2018)
- [A survey of sparse representation: algorithms and applications](https://mlsanity.com/paper/1602.07017/) (2016)
- [An Introduction to Bi-level Optimization: Foundations and Applications in Signal Processing and Machine Learning](https://mlsanity.com/paper/2308.00788/) (2023)
- [Time-Varying Convex Optimization: Time-Structured Algorithms and Applications](https://mlsanity.com/paper/2006.08500/) (2020)
- [Online Learning with Inexact Proximal Online Gradient Descent Algorithms](https://mlsanity.com/paper/1806.00202/) (2018)
- [A Survey on Nonconvex Regularization Based Sparse and Low-Rank Recovery in Signal Processing, Statistics, and Machine Learning](https://mlsanity.com/paper/1808.05403/) (2018)
- [Sparse Regularization via Convex Analysis](https://mlsanity.com/paper/1803.06765/) (2018)
- [Learning with Gradient Descent and Weakly Convex Losses](https://mlsanity.com/paper/2101.04968/) (2021)
- [Weakly Convex Regularisers for Inverse Problems: Convergence of Critical Points and Primal-Dual Optimisation](https://mlsanity.com/paper/2402.01052/) (2024)
- [Deep Weight Factorization: Sparse Learning Through the Lens of Artificial Symmetries](https://mlsanity.com/paper/2502.02496/) (2025)
- [Cascaded Channel Estimation for Large Intelligent Metasurface Assisted Massive MIMO](https://mlsanity.com/paper/1905.07948/) (2019)
- [Smoothing the Edges: Smooth Optimization for Sparse Regularization using Hadamard Overparametrization](https://mlsanity.com/paper/2307.03571/) (2023)
- [Online Proximal-ADMM For Time-varying Constrained Convex Optimization](https://mlsanity.com/paper/2005.03267/) (2020)
- [Medical Image Segmentation via Sparse Coding Decoder](https://mlsanity.com/paper/2310.10957/) (2023)
- [Monotone Lipschitz-Gradient Denoiser: Explainability of Operator Regularization Approaches Free From Lipschitz Constant Control](https://mlsanity.com/paper/2406.04676/) (2024)
- [The Proximal Operator of the Piece-wise Exponential Function and Its Application in Compressed Sensing](https://mlsanity.com/paper/2306.13425/) (2023)
- [Continuous Relaxation of Discontinuous Shrinkage Operator: Proximal Inclusion and Conversion](https://mlsanity.com/paper/2409.05316/) (2024)

## Built on by (most cited first, 1 of 1)

- [Deep Unfolded Latent Optimally Partitioned-l2/l1 Networks for Data-driven Block-Sparse Recovery](https://mlsanity.com/paper/2606.12740/) (2026)

## Abstract (arXiv)

Sparse regularization is fundamental in signal processing and feature extraction but often relies on non-differentiable penalties, conflicting with gradient-based optimizers. We propose WEEP (Weakly-convex Envelope of Piecewise Penalty), a novel differentiable regularizer derived from the weakly-convex envelope framework. WEEP provides tunable, unbiased sparsity and a simple closed-form proximal operator, while maintaining full differentiability and L-smoothness, ensuring compatibility with both gradient-based and proximal algorithms. This resolves the tradeoff between statistical performance and computational tractability. We demonstrate superior performance compared to established convex and non-convex sparse regularizers on challenging compressive sensing and image denoising tasks.