# MoDL: Model Based Deep Learning Architecture for Inverse Problems

arXiv: https://arxiv.org/abs/1712.02862 · HTML: https://mlsanity.com/paper/1712.02862/

## Key facts

- Cited by 387 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Dec 2017.

## Built on by (most cited first, 30 of 387)

- [Robust Compressed Sensing MRI with Deep Generative Priors](https://mlsanity.com/paper/2108.01368/) (2021)
- [Learning to Optimize: A Primer and A Benchmark](https://mlsanity.com/paper/2103.12828/) (2021)
- [Deep Equilibrium Architectures for Inverse Problems in Imaging](https://mlsanity.com/paper/2102.07944/) (2021)
- [Adaptive Diffusion Priors for Accelerated MRI Reconstruction](https://mlsanity.com/paper/2207.05876/) (2022)
- [Unsupervised MRI Reconstruction via Zero-Shot Learned Adversarial Transformers](https://mlsanity.com/paper/2105.08059/) (2021)
- [InverseBench: Benchmarking Plug-and-Play Diffusion Priors for Inverse Problems in Physical Sciences](https://mlsanity.com/paper/2503.11043/) (2025)
- [Direct Diffusion Bridge using Data Consistency for Inverse Problems](https://mlsanity.com/paper/2305.19809/) (2023)
- [Measurement-conditioned Denoising Diffusion Probabilistic Model for Under-sampled Medical Image Reconstruction](https://mlsanity.com/paper/2203.03623/) (2022)
- [Diffusion Model Based Posterior Sampling for Noisy Linear Inverse Problems](https://mlsanity.com/paper/2211.12343/) (2022)
- [Zero-Shot Self-Supervised Learning for MRI Reconstruction](https://mlsanity.com/paper/2102.07737/) (2021)
- [Physics-Driven Deep Learning for Computational Magnetic Resonance Imaging](https://mlsanity.com/paper/2203.12215/) (2022)
- [Recovery of Continuous 3D Refractive Index Maps from Discrete Intensity-Only Measurements using Neural Fields](https://mlsanity.com/paper/2112.00002/) (2021)
- [Bayesian MRI Reconstruction with Joint Uncertainty Estimation using Diffusion Models](https://mlsanity.com/paper/2202.01479/) (2022)
- [Multi-Modal Transformer for Accelerated MR Imaging](https://mlsanity.com/paper/2106.14248/) (2021)
- [High-Frequency Space Diffusion Models for Accelerated MRI](https://mlsanity.com/paper/2208.05481/) (2022)
- [A Neural-Network-Based Convex Regularizer for Inverse Problems](https://mlsanity.com/paper/2211.12461/) (2022)
- [Memory-Augmented Deep Unfolding Network for Compressive Sensing](https://mlsanity.com/paper/2110.09766/) (2021)
- [AI-based Reconstruction for Fast MRI -- A Systematic Review and Meta-analysis](https://mlsanity.com/paper/2112.12744/) (2021)
- [Self-Supervised MRI Reconstruction with Unrolled Diffusion Models](https://mlsanity.com/paper/2306.16654/) (2023)
- [Federated Learning of Generative Image Priors for MRI Reconstruction](https://mlsanity.com/paper/2202.04175/) (2022)
- [Ambient Diffusion Posterior Sampling: Solving Inverse Problems with Diffusion Models Trained on Corrupted Data](https://mlsanity.com/paper/2403.08728/) (2024)
- [DSFormer: A Dual-domain Self-supervised Transformer for Accelerated Multi-contrast MRI Reconstruction](https://mlsanity.com/paper/2201.10776/) (2022)
- [Bayesian Uncertainty Estimation of Learned Variational MRI Reconstruction](https://mlsanity.com/paper/2102.06665/) (2021)
- [A theoretical framework for self-supervised MR image reconstruction using sub-sampling via variable density Noisier2Noise](https://mlsanity.com/paper/2205.10278/) (2022)
- [One-dimensional Deep Low-rank and Sparse Network for Accelerated MRI](https://mlsanity.com/paper/2112.04721/) (2021)
- [Self-Supervised Learning for MRI Reconstruction with a Parallel Network Training Framework](https://mlsanity.com/paper/2109.12502/) (2021)
- [Learning a Model-Driven Variational Network for Deformable Image Registration](https://mlsanity.com/paper/2105.12227/) (2021)
- [What's in a Prior? Learned Proximal Networks for Inverse Problems](https://mlsanity.com/paper/2310.14344/) (2023)
- [IMJENSE: Scan-specific Implicit Representation for Joint Coil Sensitivity and Image Estimation in Parallel MRI](https://mlsanity.com/paper/2311.12892/) (2023)
- [Recovery Analysis for Plug-and-Play Priors using the Restricted Eigenvalue Condition](https://mlsanity.com/paper/2106.03668/) (2021)

## Abstract (arXiv)

We introduce a model-based image reconstruction framework with a convolution neural network (CNN) based regularization prior. The proposed formulation provides a systematic approach for deriving deep architectures for inverse problems with the arbitrary structure. Since the forward model is explicitly accounted for, a smaller network with fewer parameters is sufficient to capture the image information compared to black-box deep learning approaches, thus reducing the demand for training data and training time. Since we rely on end-to-end training, the CNN weights are customized to the forward model, thus offering improved performance over approaches that rely on pre-trained denoisers. The main difference of the framework from existing end-to-end training strategies is the sharing of the network weights across iterations and channels. Our experiments show that the decoupling of the number of iterations from the network complexity offered by this approach provides benefits including lower demand for training data, reduced risk of overfitting, and implementations with significantly reduced memory footprint. We propose to enforce data-consistency by using numerical optimization blocks such as conjugate gradients algorithm within the network; this approach offers faster convergence per iteration, compared to methods that rely on proximal gradients steps to enforce data consistency. Our experiments show that the faster convergence translates to improved performance, especially when the available GPU memory restricts the number of iterations.