# Momentum-Net: Fast and convergent iterative neural network for inverse problems

arXiv: https://arxiv.org/abs/1907.11818 · HTML: https://mlsanity.com/paper/1907.11818/

## Key facts

- Cited by 38 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Jul 2019.

## Built on by (most cited first, 30 of 38)

- [NeRP: Implicit Neural Representation Learning with Prior Embedding for Sparsely Sampled Image Reconstruction](https://mlsanity.com/paper/2108.10991/) (2021)
- [Deep Equilibrium Architectures for Inverse Problems in Imaging](https://mlsanity.com/paper/2102.07944/) (2021)
- [Momentum Residual Neural Networks](https://mlsanity.com/paper/2102.07870/) (2021)
- [Content-aware Scalable Deep Compressed Sensing](https://mlsanity.com/paper/2207.09313/) (2022)
- [SGD-Net: Efficient Model-Based Deep Learning with Theoretical Guarantees](https://mlsanity.com/paper/2101.09379/) (2021)
- [Patch-Based Denoising Diffusion Probabilistic Model for Sparse-View CT Reconstruction](https://mlsanity.com/paper/2211.10388/) (2022)
- [Sparse-view Cone Beam CT Reconstruction using Data-consistent Supervised and Adversarial Learning from Scarce Training Data](https://mlsanity.com/paper/2201.09318/) (2022)
- [DPER: Diffusion Prior Driven Neural Representation for Limited Angle and Sparse View CT Reconstruction](https://mlsanity.com/paper/2404.17890/) (2024)
- [Physics-/Model-Based and Data-Driven Methods for Low-Dose Computed Tomography: A survey](https://mlsanity.com/paper/2203.15725/) (2022)
- [Near-Exact Recovery for Tomographic Inverse Problems via Deep Learning](https://mlsanity.com/paper/2206.07050/) (2022)
- [PARCEL: Physics-based Unsupervised Contrastive Representation Learning for Multi-coil MR Imaging](https://mlsanity.com/paper/2202.01494/) (2022)
- [Parallel Diffusion Model-based Sparse-view Cone-beam Breast CT](https://mlsanity.com/paper/2303.12861/) (2023)
- [Provably Convergent Learned Inexact Descent Algorithm for Low-Dose CT Reconstruction](https://mlsanity.com/paper/2104.12939/) (2021)
- [RevBiFPN: The Fully Reversible Bidirectional Feature Pyramid Network](https://mlsanity.com/paper/2206.14098/) (2022)
- [Model-based Reconstruction with Learning: From Unsupervised to Supervised and Beyond](https://mlsanity.com/paper/2103.14528/) (2021)
- [Equilibrated Zeroth-Order Unrolled Deep Networks for Accelerated MRI](https://mlsanity.com/paper/2112.09891/) (2021)
- [Spectral2Spectral: Image-spectral Similarity Assisted Spectral CT Deep Reconstruction without Reference](https://mlsanity.com/paper/2210.01125/) (2022)
- [Connections between Deep Equilibrium and Sparse Representation Models with Application to Hyperspectral Image Denoising](https://mlsanity.com/paper/2203.15901/) (2022)
- [Unsupervised approaches based on optimal transport and convex analysis for inverse problems in imaging](https://mlsanity.com/paper/2311.08972/) (2023)
- [Low-Dose CT Using Denoising Diffusion Probabilistic Model for 20× Speedup](https://mlsanity.com/paper/2209.15136/) (2022)
- [Solving Inverse Problems with Model Mismatch using Untrained Neural Networks within Model-based Architectures](https://mlsanity.com/paper/2403.04847/) (2024)
- [Swap-Net: A Memory-Efficient 2.5D Network for Sparse-View 3D Cone Beam CT Reconstruction](https://mlsanity.com/paper/2410.10836/) (2024)
- [Progressive Fusion for Multimodal Integration](https://mlsanity.com/paper/2209.00302/) (2022)
- [Deep Radon Prior: A Fully Unsupervised Framework for Sparse-View CT Reconstruction](https://mlsanity.com/paper/2401.00135/) (2023)
- [Fed-ADE: Adaptive Learning Rate for Federated Post-adaptation under Distribution Shift](https://mlsanity.com/paper/2603.01040/) (2026)
- [Convolutional Analysis Operator Learning by End-To-End Training of Iterative Neural Networks](https://mlsanity.com/paper/2203.02166/) (2022)
- [Self-supervised regression learning using domain knowledge: Applications to improving self-supervised denoising in imaging](https://mlsanity.com/paper/2205.04821/) (2022)
- [Unrolled algorithms for group synchronization](https://mlsanity.com/paper/2207.09418/) (2022)
- [Physics-Driven ML-Based Modelling for Correcting Inverse Estimation](https://mlsanity.com/paper/2309.13985/) (2023)
- [AI-Enabled Ultra-Low-Dose CT Reconstruction](https://mlsanity.com/paper/2106.09834/) (2021)

## Abstract (arXiv)

Iterative neural networks (INN) are rapidly gaining attention for solving inverse problems in imaging, image processing, and computer vision. INNs combine regression NNs and an iterative model-based image reconstruction (MBIR) algorithm, often leading to both good generalization capability and outperforming reconstruction quality over existing MBIR optimization models. This paper proposes the first fast and convergent INN architecture, Momentum-Net, by generalizing a block-wise MBIR algorithm that uses momentum and majorizers with regression NNs. For fast MBIR, Momentum-Net uses momentum terms in extrapolation modules, and noniterative MBIR modules at each iteration by using majorizers, where each iteration of Momentum-Net consists of three core modules: image refining, extrapolation, and MBIR. Momentum-Net guarantees convergence to a fixed-point for general differentiable (non)convex MBIR functions (or data-fit terms) and convex feasible sets, under two asymptomatic conditions. To consider data-fit variations across training and testing samples, we also propose a regularization parameter selection scheme based on the "spectral spread" of majorization matrices. Numerical experiments for light-field photography using a focal stack and sparse-view computational tomography demonstrate that, given identical regression NN architectures, Momentum-Net significantly improves MBIR speed and accuracy over several existing INNs; it significantly improves reconstruction quality compared to a state-of-the-art MBIR method in each application.