# Gradient-Based Learning of Discrete Structured Measurement Operators for Signal Recovery

arXiv: https://arxiv.org/abs/2202.03391 · HTML: https://mlsanity.com/paper/2202.03391/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 22 of its 71 references that are in the map.
- First posted on arXiv 7 Feb 2022.

## Builds on (most cited first, 22 of 22)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016)
- [Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation](https://mlsanity.com/paper/1308.3432/) (2013)
- [The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables](https://mlsanity.com/paper/1611.00712/) (2016)
- [Algorithm Unrolling: Interpretable, Efficient Deep Learning for Signal and Image Processing](https://mlsanity.com/paper/1912.10557/) (2019)
- [SNAS: Stochastic Neural Architecture Search](https://mlsanity.com/paper/1812.09926/) (2018)
- [A* Sampling](https://mlsanity.com/paper/1411.0030/) (2014)
- [Neural Variational Inference and Learning in Belief Networks](https://mlsanity.com/paper/1402.0030/) (2014)
- [Theoretical Linear Convergence of Unfolded ISTA and its Practical Weights and Thresholds](https://mlsanity.com/paper/1808.10038/) (2018)
- [Advancing machine learning for MR image reconstruction with an open competition: Overview of the 2019 fastMRI challenge](https://mlsanity.com/paper/2001.02518/) (2020)
- [Solving Inverse Problems With Deep Neural Networks -- Robustness Included?](https://mlsanity.com/paper/2011.04268/) (2020)
- [GANS for Sequences of Discrete Elements with the Gumbel-softmax Distribution](https://mlsanity.com/paper/1611.04051/) (2016)
- [Learning-based Optimization of the Under-sampling Pattern in MRI](https://mlsanity.com/paper/1901.01960/) (2019)
- [Learning a Compressed Sensing Measurement Matrix via Gradient Unrolling](https://mlsanity.com/paper/1806.10175/) (2018)
- [Results of the 2020 fastMRI Challenge for Machine Learning MR Image Reconstruction](https://mlsanity.com/paper/2012.06318/) (2020)
- [Learning Sampling and Model-Based Signal Recovery for Compressed Sensing MRI](https://mlsanity.com/paper/2004.10536/) (2020)
- [Compressed Learning: A Deep Neural Network Approach](https://mlsanity.com/paper/1610.09615/) (2016)
- [Neurally Augmented ALISTA](https://mlsanity.com/paper/2010.01930/) (2020)
- [Plug-And-Play Learned Gaussian-mixture Approximate Message Passing](https://mlsanity.com/paper/2011.09388/) (2020)
- [Deep-learning-based Optimization of the Under-sampling Pattern in MRI](https://mlsanity.com/paper/1907.11374/) (2019)

## Abstract (arXiv)

Countless signal processing applications include the reconstruction of signals from few indirect linear measurements. The design of effective measurement operators is typically constrained by the underlying hardware and physics, posing a challenging and often even discrete optimization task. While the potential of gradient-based learning via the unrolling of iterative recovery algorithms has been demonstrated, it has remained unclear how to leverage this technique when the set of admissible measurement operators is structured and discrete. We tackle this problem by combining unrolled optimization with Gumbel reparametrizations, which enable the computation of low-variance gradient estimates of categorical random variables. Our approach is formalized by GLODISMO (Gradient-based Learning of DIscrete Structured Measurement Operators). This novel method is easy-to-implement, computationally efficient, and extendable due to its compatibility with automatic differentiation. We empirically demonstrate the performance and flexibility of GLODISMO in several prototypical signal recovery applications, verifying that the learned measurement matrices outperform conventional designs based on randomization as well as discrete optimization baselines.