# Deep End-to-End Posterior ENergy (DEEPEN) for image recovery

arXiv: https://arxiv.org/abs/2503.17244 · HTML: https://mlsanity.com/paper/2503.17244/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Inverse problems and image restoration, one of 14 directions in Diffusion models.
- Builds on 22 of its 37 references that are in the map.
- First posted on arXiv 21 Mar 2025.

## Placed in

- Diffusion models / [Inverse problems and image restoration](https://mlsanity.com/diffusion/inverse-problems-and-image-restoration/)

## Builds on (most cited first, 22 of 22)

- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Diffusion Posterior Sampling for General Noisy Inverse Problems](https://mlsanity.com/paper/2209.14687/) (2022)
- [Deep Equilibrium Models](https://mlsanity.com/paper/1909.01377/) (2019)
- [fastMRI: An Open Dataset and Benchmarks for Accelerated MRI](https://mlsanity.com/paper/1811.08839/) (2018)
- [Algorithm Unrolling: Interpretable, Efficient Deep Learning for Signal and Image Processing](https://mlsanity.com/paper/1912.10557/) (2019)
- [MoDL: Model Based Deep Learning Architecture for Inverse Problems](https://mlsanity.com/paper/1712.02862/) (2017)
- [Learning a Variational Network for Reconstruction of Accelerated MRI Data](https://mlsanity.com/paper/1704.00447/) (2017)
- [How to Train Your Energy-Based Models](https://mlsanity.com/paper/2101.03288/) (2021)
- [Plug-and-Play Methods Provably Converge with Properly Trained Denoisers](https://mlsanity.com/paper/1905.05406/) (2019)
- [Deep Equilibrium Architectures for Inverse Problems in Imaging](https://mlsanity.com/paper/2102.07944/) (2021)
- [Gradient Step Denoiser for convergent Plug-and-Play](https://mlsanity.com/paper/2110.03220/) (2021)
- [On the Anatomy of MCMC-Based Maximum Likelihood Learning of Energy-Based Models](https://mlsanity.com/paper/1903.12370/) (2019)
- [PnP-Flow: Plug-and-Play Image Restoration with Flow Matching](https://mlsanity.com/paper/2410.02423/) (2024)
- [Plug and play methods for magnetic resonance imaging (long version)](https://mlsanity.com/paper/1903.08616/) (2019)
- [Plug-and-Play Unplugged: Optimization Free Reconstruction using Consensus Equilibrium](https://mlsanity.com/paper/1705.08983/) (2017)
- [MADGAN: unsupervised Medical Anomaly Detection GAN using multiple adjacent brain MRI slice reconstruction](https://mlsanity.com/paper/2007.13559/) (2020)
- [CLIP: Cheap Lipschitz Training of Neural Networks](https://mlsanity.com/paper/2103.12531/) (2021)
- [Learning Energy-Based Models in High-Dimensional Spaces with Multi-scale Denoising Score Matching](https://mlsanity.com/paper/1910.07762/) (2019)
- [Multi-Scale Energy (MuSE) plug and play framework for inverse problems](https://mlsanity.com/paper/2305.04775/) (2023)
- [Memory-efficient model-based deep learning with convergence and robustness guarantees](https://mlsanity.com/paper/2206.04797/) (2022)
- [Memory-efficient deep end-to-end posterior network (DEEPEN) for inverse problems](https://mlsanity.com/paper/2402.05422/) (2024)

## Abstract (arXiv)

Current end-to-end (E2E) and plug-and-play (PnP) image reconstruction algorithms approximate the maximum a posteriori (MAP) estimate but cannot offer sampling from the posterior distribution, like diffusion models. By contrast, it is challenging for diffusion models to be trained in an E2E fashion. This paper introduces a Deep End-to-End Posterior ENergy (DEEPEN) framework, which enables MAP estimation as well as sampling. We learn the parameters of the posterior, which is the sum of the data consistency error and the negative log-prior distribution, using maximum likelihood optimization in an E2E fashion. The proposed approach does not require algorithm unrolling, and hence has a smaller computational and memory footprint than current E2E methods, while it does not require contraction constraints typically needed by current PnP methods. Our results demonstrate that DEEPEN offers improved performance than current E2E and PnP models in the MAP setting, while it also offers faster sampling compared to diffusion models. In addition, the learned energy-based model is observed to be more robust to changes in image acquisition settings.