# Connections between Deep Equilibrium and Sparse Representation Models with Application to Hyperspectral Image Denoising

arXiv: https://arxiv.org/abs/2203.15901 · HTML: https://mlsanity.com/paper/2203.15901/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 78 references that are in the map.
- First posted on arXiv 29 Mar 2022.

## Builds on (most cited first, 24 of 24)

- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Spectral Normalization for Generative Adversarial Networks](https://mlsanity.com/paper/1802.05957/) (2018)
- [Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising](https://mlsanity.com/paper/1608.03981/) (2016)
- [Deep Equilibrium Models](https://mlsanity.com/paper/1909.01377/) (2019)
- [Deep Convolutional Neural Network for Inverse Problems in Imaging](https://mlsanity.com/paper/1611.03679/) (2016)
- [Algorithm Unrolling: Interpretable, Efficient Deep Learning for Signal and Image Processing](https://mlsanity.com/paper/1912.10557/) (2019)
- [Learning Deep CNN Denoiser Prior for Image Restoration](https://mlsanity.com/paper/1704.03264/) (2017)
- [Deep Learning Techniques for Inverse Problems in Imaging](https://mlsanity.com/paper/2005.06001/) (2020)
- [MemNet: A Persistent Memory Network for Image Restoration](https://mlsanity.com/paper/1708.02209/) (2017)
- [Plug-and-Play Methods Provably Converge with Properly Trained Denoisers](https://mlsanity.com/paper/1905.05406/) (2019)
- [Deep Equilibrium Architectures for Inverse Problems in Imaging](https://mlsanity.com/paper/2102.07944/) (2021)
- [Efficient tensor completion for color image and video recovery: Low-rank tensor train](https://mlsanity.com/paper/1606.01500/) (2016)
- [Hyperspectral Image Restoration via Total Variation Regularized Low-rank Tensor Decomposition](https://mlsanity.com/paper/1707.02477/) (2017)
- [Trellis Networks for Sequence Modeling](https://mlsanity.com/paper/1810.06682/) (2018)
- [Hyperspectral Image Denoising Employing a Spatial-Spectral Deep Residual Convolutional Neural Network](https://mlsanity.com/paper/1806.00183/) (2018)
- [3D Quasi-Recurrent Neural Network for Hyperspectral Image Denoising](https://mlsanity.com/paper/2003.04547/) (2020)
- [Fast Hyperspectral Image Denoising and Inpainting Based on Low-Rank and Sparse Representations](https://mlsanity.com/paper/2103.06842/) (2021)
- [Recurrent Stacking of Layers for Compact Neural Machine Translation Models](https://mlsanity.com/paper/1807.05353/) (2018)
- [Momentum-Net: Fast and convergent iterative neural network for inverse problems](https://mlsanity.com/paper/1907.11818/) (2019)
- [Deep K-SVD Denoising](https://mlsanity.com/paper/1909.13164/) (2019)
- [Image Restoration for Remote Sensing: Overview and Toolbox](https://mlsanity.com/paper/2107.00557/) (2021)
- [Denoising Hyperspectral Image with Non-i.i.d. Noise Structure](https://mlsanity.com/paper/1702.00098/) (2017)
- [Model Adaptation for Inverse Problems in Imaging](https://mlsanity.com/paper/2012.00139/) (2020)
- [Learned Greedy Method (LGM): A Novel Neural Architecture for Sparse Coding and Beyond](https://mlsanity.com/paper/2010.07069/) (2020)

## Built on by (most cited first, 4 of 4)

- [Positive concave deep equilibrium models](https://mlsanity.com/paper/2402.04029/) (2024)
- [Deep Equilibrium Models Meet Federated Learning](https://mlsanity.com/paper/2305.18646/) (2023)
- [A Deep Unrolling Model with Hybrid Optimization Structure for Hyperspectral Image Deconvolution](https://mlsanity.com/paper/2306.06378/) (2023)
- [Deep Equilibrium Convolutional Sparse Coding for Hyperspectral Image Denoising](https://mlsanity.com/paper/2508.15553/) (2025)

## Abstract (arXiv)

In this study, the problem of computing a sparse representation of multi-dimensional visual data is considered. In general, such data e.g., hyperspectral images, color images or video data consists of signals that exhibit strong local dependencies. A new computationally efficient sparse coding optimization problem is derived by employing regularization terms that are adapted to the properties of the signals of interest. Exploiting the merits of the learnable regularization techniques, a neural network is employed to act as structure prior and reveal the underlying signal dependencies. To solve the optimization problem Deep unrolling and Deep equilibrium based algorithms are developed, forming highly interpretable and concise deep-learning-based architectures, that process the input dataset in a block-by-block fashion. Extensive simulation results, in the context of hyperspectral image denoising, are provided, which demonstrate that the proposed algorithms outperform significantly other sparse coding approaches and exhibit superior performance against recent state-of-the-art deep-learning-based denoising models. In a wider perspective, our work provides a unique bridge between a classic approach, that is the sparse representation theory, and modern representation tools that are based on deep learning modeling.