# Quantized Proximal Averaging Network for Analysis Sparse Coding

arXiv: https://arxiv.org/abs/2105.06211 · HTML: https://mlsanity.com/paper/2105.06211/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 14 of its 45 references that are in the map.
- First posted on arXiv 13 May 2021.

## Builds on (most cited first, 14 of 14)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks](https://mlsanity.com/paper/1603.05279/) (2016)
- [BinaryConnect: Training Deep Neural Networks with binary weights during propagations](https://mlsanity.com/paper/1511.00363/) (2015)
- [Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activations](https://mlsanity.com/paper/1609.07061/) (2016)
- [Learning Deep CNN Denoiser Prior for Image Restoration](https://mlsanity.com/paper/1704.03264/) (2017)
- [LQ-Nets: Learned Quantization for Highly Accurate and Compact Deep Neural Networks](https://mlsanity.com/paper/1807.10029/) (2018)
- [Bi-Real Net: Enhancing the Performance of 1-bit CNNs With Improved Representational Capability and Advanced Training Algorithm](https://mlsanity.com/paper/1808.00278/) (2018)
- [From Denoising to Compressed Sensing](https://mlsanity.com/paper/1406.4175/) (2014)
- [A Deep Learning Approach to Structured Signal Recovery](https://mlsanity.com/paper/1508.04065/) (2015)
- [Group-based Sparse Representation for Image Restoration](https://mlsanity.com/paper/1405.3351/) (2014)
- [Training Binary Neural Networks using the Bayesian Learning Rule](https://mlsanity.com/paper/2002.10778/) (2020)
- [QGAN: Quantized Generative Adversarial Networks](https://mlsanity.com/paper/1901.08263/) (2019)
- [Image Compressive Sensing Recovery Using Adaptively Learned Sparsifying Basis via L0 Minimization](https://mlsanity.com/paper/1404.7566/) (2014)
- [NASB: Neural Architecture Search for Binary Convolutional Neural Networks](https://mlsanity.com/paper/2008.03515/) (2020)

## Abstract (arXiv)

We solve the analysis sparse coding problem considering a combination of convex and non-convex sparsity promoting penalties. The multi-penalty formulation results in an iterative algorithm involving proximal-averaging. We then unfold the iterative algorithm into a trainable network that facilitates learning the sparsity prior. We also consider quantization of the network weights. Quantization makes neural networks efficient both in terms of memory and computation during inference, and also renders them compatible for low-precision hardware deployment. Our learning algorithm is based on a variant of the ADAM optimizer in which the quantizer is part of the forward pass and the gradients of the loss function are evaluated corresponding to the quantized weights while doing a book-keeping of the high-precision weights. We demonstrate applications to compressed image recovery and magnetic resonance image reconstruction. The proposed approach offers superior reconstruction accuracy and quality than state-of-the-art unfolding techniques and the performance degradation is minimal even when the weights are subjected to extreme quantization.