# Deep Unfolding: Model-Based Inspiration of Novel Deep Architectures

arXiv: https://arxiv.org/abs/1409.2574 · HTML: https://mlsanity.com/paper/1409.2574/

## Key facts

- Cited by 77 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Sep 2014.

## Built on by (most cited first, 30 of 77)

- [Learning to Optimize: A Primer and A Benchmark](https://mlsanity.com/paper/2103.12828/) (2021)
- [Graph Neural Networks Inspired by Classical Iterative Algorithms](https://mlsanity.com/paper/2103.06064/) (2021)
- [Equivariant Hypergraph Diffusion Neural Operators](https://mlsanity.com/paper/2207.06680/) (2022)
- [On the Explainability of Natural Language Processing Deep Models](https://mlsanity.com/paper/2210.06929/) (2022)
- [Transformers from an Optimization Perspective](https://mlsanity.com/paper/2205.13891/) (2022)
- [Stable and Interpretable Unrolled Dictionary Learning](https://mlsanity.com/paper/2106.00058/) (2021)
- [Implicit vs Unfolded Graph Neural Networks](https://mlsanity.com/paper/2111.06592/) (2021)
- [Learning to Continuously Optimize Wireless Resource in a Dynamic Environment: A Bilevel Optimization Perspective](https://mlsanity.com/paper/2105.01696/) (2021)
- [LoRD-Net: Unfolded Deep Detection Network with Low-Resolution Receivers](https://mlsanity.com/paper/2102.02993/) (2021)
- [Robust Stochastically-Descending Unrolled Networks](https://mlsanity.com/paper/2312.15788/) (2023)
- [Model-Based Machine Learning for Communications](https://mlsanity.com/paper/2101.04726/) (2021)
- [Edge-Aware Autoencoder Design for Real-Time Mixture-of-Experts Image Compression](https://mlsanity.com/paper/2207.12348/) (2022)
- [Efficient Deep Unfolding for SISO-OFDM Channel Estimation](https://mlsanity.com/paper/2210.06588/) (2022)
- [Deep Unfolding: Recent Developments, Theory, and Design Guidelines](https://mlsanity.com/paper/2512.03768/) (2025)
- [Deep Graph Unfolding for Beamforming in MU-MIMO Interference Networks](https://mlsanity.com/paper/2304.00446/) (2023)
- [FAID Diversity via Neural Networks](https://mlsanity.com/paper/2105.04118/) (2021)
- [Graph-based Algorithm Unfolding for Energy-aware Power Allocation in Wireless Networks](https://mlsanity.com/paper/2201.11799/) (2022)
- [DeDA: Deep Directed Accumulator](https://mlsanity.com/paper/2303.08434/) (2023)
- [Dynamic Proximal Unrolling Network for Compressive Imaging](https://mlsanity.com/paper/2107.11007/) (2021)
- [Deep Learning for Estimation and Pilot Signal Design in Few-Bit Massive MIMO Systems](https://mlsanity.com/paper/2107.11958/) (2021)
- [Modeling Mask Uncertainty in Hyperspectral Image Reconstruction](https://mlsanity.com/paper/2112.15362/) (2021)
- [Spectral Compressive Imaging Reconstruction Using Convolution and Contextual Transformer](https://mlsanity.com/paper/2201.05768/) (2022)
- [DECONET: an Unfolding Network for Analysis-based Compressed Sensing with Generalization Error Bounds](https://mlsanity.com/paper/2205.07050/) (2022)
- [Deep Learning Based Uplink Multi-User SIMO Beamforming Design](https://mlsanity.com/paper/2309.16603/) (2023)
- [Unveiling and Mitigating Adversarial Vulnerabilities in Iterative Optimizers](https://mlsanity.com/paper/2504.19000/) (2025)
- [Deep FlexQP: Accelerated Nonlinear Programming via Deep Unfolding](https://mlsanity.com/paper/2512.01565/) (2025)
- [On the convergence of group-sparse autoencoders](https://mlsanity.com/paper/2102.07003/) (2021)
- [Unrolling PALM for sparse semi-blind source separation](https://mlsanity.com/paper/2112.05694/) (2021)
- [Tight integration of neural- and clustering-based diarization through deep unfolding of infinite Gaussian mixture model](https://mlsanity.com/paper/2202.06524/) (2022)
- [Block Modulating Video Compression: An Ultra Low Complexity Image Compression Encoder for Resource Limited Platforms](https://mlsanity.com/paper/2205.03677/) (2022)

## Abstract (arXiv)

Model-based methods and deep neural networks have both been tremendously successful paradigms in machine learning. In model-based methods, problem domain knowledge can be built into the constraints of the model, typically at the expense of difficulties during inference. In contrast, deterministic deep neural networks are constructed in such a way that inference is straightforward, but their architectures are generic and it is unclear how to incorporate knowledge. This work aims to obtain the advantages of both approaches. To do so, we start with a model-based approach and an associated inference algorithm, and unfold the inference iterations as layers in a deep network. Rather than optimizing the original model, we untie the model parameters across layers, in order to create a more powerful network. The resulting architecture can be trained discriminatively to perform accurate inference within a fixed network size. We show how this framework allows us to interpret conventional networks as mean-field inference in Markov random fields, and to obtain new architectures by instead using belief propagation as the inference algorithm. We then show its application to a non-negative matrix factorization model that incorporates the problem-domain knowledge that sound sources are additive. Deep unfolding of this model yields a new kind of non-negative deep neural network, that can be trained using a multiplicative backpropagation-style update algorithm. We present speech enhancement experiments showing that our approach is competitive with conventional neural networks despite using far fewer parameters.