# Understanding approximate and unrolled dictionary learning for pattern recovery

arXiv: https://arxiv.org/abs/2106.06338 · HTML: https://mlsanity.com/paper/2106.06338/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 40 references that are in the map.
- First posted on arXiv 11 Jun 2021.

## Builds on (most cited first, 16 of 16)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Visualizing the Loss Landscape of Neural Nets](https://mlsanity.com/paper/1712.09913/) (2017)
- [Truncated Back-propagation for Bilevel Optimization](https://mlsanity.com/paper/1810.10667/) (2018)
- [Painless Stochastic Gradient: Interpolation, Line-Search, and Convergence Rates](https://mlsanity.com/paper/1905.09997/) (2019)
- [Theoretical Linear Convergence of Unfolded ISTA and its Practical Weights and Thresholds](https://mlsanity.com/paper/1808.10038/) (2018)
- [Complete Dictionary Recovery over the Sphere I: Overview and the Geometric Picture](https://mlsanity.com/paper/1511.03607/) (2015)
- [Implicit differentiation of Lasso-type models for hyperparameter optimization](https://mlsanity.com/paper/2002.08943/) (2020)
- [Learning step sizes for unfolded sparse coding](https://mlsanity.com/paper/1905.11071/) (2019)
- [Deep K-SVD Denoising](https://mlsanity.com/paper/1909.13164/) (2019)
- [Learning Sparsely Used Overcomplete Dictionaries via Alternating Minimization](https://mlsanity.com/paper/1310.7991/) (2013)
- [Super-efficiency of automatic differentiation for functions defined as a minimum](https://mlsanity.com/paper/2002.03722/) (2020)
- [Global Optimality in Tensor Factorization, Deep Learning, and Beyond](https://mlsanity.com/paper/1506.07540/) (2015)
- [Deep Network Classification by Scattering and Homotopy Dictionary Learning](https://mlsanity.com/paper/1910.03561/) (2019)
- [Sparse and spurious: dictionary learning with noise and outliers](https://mlsanity.com/paper/1407.5155/) (2014)
- [Multivariate Convolutional Sparse Coding for Electromagnetic Brain Signals](https://mlsanity.com/paper/1805.09654/) (2018)
- [Deep Transform and Metric Learning Network: Wedding Deep Dictionary Learning and Neural Networks](https://mlsanity.com/paper/2002.07898/) (2020)

## Built on by (most cited first, 11 of 11)

- [From Flat to Hierarchical: Extracting Sparse Representations with Matching Pursuit](https://mlsanity.com/paper/2506.03093/) (2025)
- [WPPNets and WPPFlows: The Power of Wasserstein Patch Priors for Superresolution](https://mlsanity.com/paper/2201.08157/) (2022)
- [AbsTopK: Rethinking Sparse Autoencoders For Bidirectional Features](https://mlsanity.com/paper/2510.00404/) (2025)
- [Simple Alternating Minimization Provably Solves Complete Dictionary Learning](https://mlsanity.com/paper/2210.12816/) (2022)
- [Learning Variational Models with Unrolling and Bilevel Optimization](https://mlsanity.com/paper/2209.12651/) (2022)
- [Learning of Patch-Based Smooth-Plus-Sparse Models for Image Reconstruction](https://mlsanity.com/paper/2412.13070/) (2024)
- [Mechanistic Interpretability with Sparse Autoencoder Neural Operators](https://mlsanity.com/paper/2509.03738/) (2025)
- [Probabilistic Unrolling: Scalable, Inverse-Free Maximum Likelihood Estimation for Latent Gaussian Models](https://mlsanity.com/paper/2306.03249/) (2023)
- [Evaluating Sparse Autoencoders: From Shallow Design to Matching Pursuit](https://mlsanity.com/paper/2506.05239/) (2025)
- [Meta-Prior: Meta learning for Adaptive Inverse Problem Solvers](https://mlsanity.com/paper/2311.18710/) (2023)
- [Enhancing Performance of Explainable AI Models with Constrained Concept Refinement](https://mlsanity.com/paper/2502.06775/) (2025)

## Abstract (arXiv)

Dictionary learning consists of finding a sparse representation from noisy data and is a common way to encode data-driven prior knowledge on signals. Alternating minimization (AM) is standard for the underlying optimization, where gradient descent steps alternate with sparse coding procedures. The major drawback of this method is its prohibitive computational cost, making it unpractical on large real-world data sets. This work studies an approximate formulation of dictionary learning based on unrolling and compares it to alternating minimization to find the best trade-off between speed and precision. We analyze the asymptotic behavior and convergence rate of gradients estimates in both methods. We show that unrolling performs better on the support of the inner problem solution and during the first iterations. Finally, we apply unrolling on pattern learning in magnetoencephalography (MEG) with the help of a stochastic algorithm and compare the performance to a state-of-the-art method.