# A Unified Framework for Sparse Relaxed Regularized Regression: SR3

arXiv: https://arxiv.org/abs/1807.05411 · HTML: https://mlsanity.com/paper/1807.05411/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Jul 2018.

## Built on by (most cited first, 30 of 36)

- [Learning Sparse Nonlinear Dynamics via Mixed-Integer Optimization](https://mlsanity.com/paper/2206.00176/) (2022)
- [Bayesian autoencoders for data-driven discovery of coordinates, governing equations and fundamental constants](https://mlsanity.com/paper/2211.10575/) (2022)
- [DySMHO: Data-Driven Discovery of Governing Equations for Dynamical Systems via Moving Horizon Optimization](https://mlsanity.com/paper/2108.00069/) (2021)
- [Benchmarking sparse system identification with low-dimensional chaos](https://mlsanity.com/paper/2302.10787/) (2023)
- [Convergence of uncertainty estimates in Ensemble and Bayesian sparse model discovery](https://mlsanity.com/paper/2301.12649/) (2023)
- [Rapid Bayesian identification of sparse nonlinear dynamics from scarce and noisy data](https://mlsanity.com/paper/2402.15357/) (2024)
- [Noise-aware Physics-informed Machine Learning for Robust PDE Discovery](https://mlsanity.com/paper/2206.12901/) (2022)
- [HyperSINDy: Deep Generative Modeling of Nonlinear Stochastic Governing Equations](https://mlsanity.com/paper/2310.04832/) (2023)
- [Parametric PDE Control with Deep Reinforcement Learning and Differentiable L0-Sparse Polynomial Policies](https://mlsanity.com/paper/2403.15267/) (2024)
- [On Sparse High-Dimensional Graphical Model Learning For Dependent Time Series](https://mlsanity.com/paper/2111.07897/) (2021)
- [Adaptive Uncertainty-Guided Model Selection for Data-Driven PDE Discovery](https://mlsanity.com/paper/2308.10283/) (2023)
- [BINDy -- Bayesian identification of nonlinear dynamics with reversible-jump Markov-chain Monte-Carlo](https://mlsanity.com/paper/2408.08062/) (2024)
- [Robust Trimmed k-means](https://mlsanity.com/paper/2108.07186/) (2021)
- [Adaptive parameters identification for nonlinear dynamics using deep permutation invariant networks](https://mlsanity.com/paper/2501.11350/) (2025)
- [Learning nonparametric ordinary differential equations from noisy data](https://mlsanity.com/paper/2206.15215/) (2022)
- [LES-SINDy: Laplace-Enhanced Sparse Identification of Nonlinear Dynamical Systems](https://mlsanity.com/paper/2411.01719/) (2024)
- [SINDyG: Sparse Identification of Nonlinear Dynamical Systems from Graph-Structured Data, with Applications to Stuart-Landau Oscillator Networks](https://mlsanity.com/paper/2409.04463/) (2024)
- [Discovering equations from data: symbolic regression in dynamical systems](https://mlsanity.com/paper/2508.20257/) (2025)
- [Data-driven Discovery of Digital Twins in Biomedical Research](https://mlsanity.com/paper/2508.21484/) (2025)
- [Data-driven sparse identification of governing PDEs via knockoff filters and multi-criteria trade-offs](https://mlsanity.com/paper/2605.26631/) (2026)
- [Uncovering Closed-form Governing Equations of Nonlinear Dynamics from Videos](https://mlsanity.com/paper/2106.04776/) (2021)
- [Fully differentiable model discovery](https://mlsanity.com/paper/2106.04886/) (2021)
- [SODAs: Sparse Optimization for the Discovery of Differential and Algebraic Equations](https://mlsanity.com/paper/2503.05993/) (2025)
- [Hybrid Adaptive Modeling in Process Monitoring: Leveraging Sequence Encoders and Physics-Informed Neural Networks](https://mlsanity.com/paper/2505.14252/) (2025)
- [Improved identification accuracy in equation learning via comprehensive boldsymbolR2-elimination and Bayesian model selection](https://mlsanity.com/paper/2311.13265/) (2023)
- [Sparse identification of quasipotentials via a combined data-driven method](https://mlsanity.com/paper/2407.05050/) (2024)
- [Enhancing generalizability of model discovery across parameter space with multi-experiment equation learning (ME-EQL)](https://mlsanity.com/paper/2506.08916/) (2025)
- [Sparse-mode Dynamic Mode Decomposition for Disambiguating Local and Global Structures](https://mlsanity.com/paper/2507.19787/) (2025)
- [Probabilistic and nonlinear compressive sensing](https://mlsanity.com/paper/2509.15060/) (2025)
- [A joint optimization approach to identifying sparse dynamics using least squares kernel collocation](https://mlsanity.com/paper/2511.18555/) (2025)

## Abstract (arXiv)

Regularized regression problems are ubiquitous in statistical modeling, signal processing, and machine learning. Sparse regression in particular has been instrumental in scientific model discovery, including compressed sensing applications, variable selection, and high-dimensional analysis. We propose a broad framework for sparse relaxed regularized regression, called SR3. The key idea is to solve a relaxation of the regularized problem, which has three advantages over the state-of-the-art: (1) solutions of the relaxed problem are superior with respect to errors, false positives, and conditioning, (2) relaxation allows extremely fast algorithms for both convex and nonconvex formulations, and (3) the methods apply to composite regularizers such as total variation (TV) and its nonconvex variants. We demonstrate the advantages of SR3 (computational efficiency, higher accuracy, faster convergence rates, greater flexibility) across a range of regularized regression problems with synthetic and real data, including applications in compressed sensing, LASSO, matrix completion, TV regularization, and group sparsity. To promote reproducible research, we also provide a companion MATLAB package that implements these examples.