# Computing Multiple Image Reconstructions with a Single Hypernetwork

arXiv: https://arxiv.org/abs/2202.11009 · HTML: https://mlsanity.com/paper/2202.11009/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 55 references that are in the map.
- First posted on arXiv 22 Feb 2022.

## Builds on (most cited first, 30 of 31)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics](https://mlsanity.com/paper/1705.07115/) (2017)
- [Deep Image Prior](https://mlsanity.com/paper/1711.10925/) (2017)
- [HyperNetworks](https://mlsanity.com/paper/1609.09106/) (2016)
- [A Tutorial on Bayesian Optimization](https://mlsanity.com/paper/1807.02811/) (2018)
- [VoxelMorph: A Learning Framework for Deformable Medical Image Registration](https://mlsanity.com/paper/1809.05231/) (2018)
- [Generalised Dice overlap as a deep learning loss function for highly unbalanced segmentations](https://mlsanity.com/paper/1707.03237/) (2017)
- [fastMRI: An Open Dataset and Benchmarks for Accelerated MRI](https://mlsanity.com/paper/1811.08839/) (2018)
- [Deep Convolutional Neural Network for Inverse Problems in Imaging](https://mlsanity.com/paper/1611.03679/) (2016)
- [Algorithm Unrolling: Interpretable, Efficient Deep Learning for Signal and Image Processing](https://mlsanity.com/paper/1912.10557/) (2019)
- [Parameter-efficient Multi-task Fine-tuning for Transformers via Shared Hypernetworks](https://mlsanity.com/paper/2106.04489/) (2021)
- [SMASH: One-Shot Model Architecture Search through HyperNetworks](https://mlsanity.com/paper/1708.05344/) (2017)
- [Graph HyperNetworks for Neural Architecture Search](https://mlsanity.com/paper/1810.05749/) (2018)
- [Deep Decoder: Concise Image Representations from Untrained Non-convolutional Networks](https://mlsanity.com/paper/1810.03982/) (2018)
- [A deep convolutional neural network using directional wavelets for low-dose X-ray CT reconstruction](https://mlsanity.com/paper/1610.09736/) (2016)
- [HyperMorph: Amortized Hyperparameter Learning for Image Registration](https://mlsanity.com/paper/2101.01035/) (2021)
- [Stochastic Hyperparameter Optimization through Hypernetworks](https://mlsanity.com/paper/1802.09419/) (2018)
- [Bayesian Hypernetworks](https://mlsanity.com/paper/1710.04759/) (2017)
- [Image Restoration using Total Variation Regularized Deep Image Prior](https://mlsanity.com/paper/1810.12864/) (2018)
- [Hypernetwork functional image representation](https://mlsanity.com/paper/1902.10404/) (2019)
- [Controllable Pareto Multi-Task Learning](https://mlsanity.com/paper/2010.06313/) (2020)
- [Loss Functions for Neural Networks for Image Processing](https://mlsanity.com/paper/1511.08861/) (2015)
- [Amortized Inference Regularization](https://mlsanity.com/paper/1805.08913/) (2018)
- [Accelerating Human-in-the-loop Machine Learning: Challenges and Opportunities](https://mlsanity.com/paper/1804.05892/) (2018)
- [Neural Network-based Reconstruction in Compressed Sensing MRI Without Fully-sampled Training Data](https://mlsanity.com/paper/2007.14979/) (2020)
- [HyperST-Net: Hypernetworks for Spatio-Temporal Forecasting](https://mlsanity.com/paper/1809.10889/) (2018)
- [Optimization Methods for Inverse Problems](https://mlsanity.com/paper/1712.00154/) (2017)
- [Meta Networks for Neural Style Transfer](https://mlsanity.com/paper/1709.04111/) (2017)
- [Optimization methods for MR image reconstruction (long version)](https://mlsanity.com/paper/1903.03510/) (2019)

## Built on by (most cited first, 8 of 8)

- [A Robust and Interpretable Deep Learning Framework for Multi-modal Registration via Keypoints](https://mlsanity.com/paper/2304.09941/) (2023)
- [Neural Pre-Processing: A Learning Framework for End-to-end Brain MRI Pre-processing](https://mlsanity.com/paper/2303.12148/) (2023)
- [Generalizing Supervised Deep Learning MRI Reconstruction to Multiple and Unseen Contrasts using Meta-Learning Hypernetworks](https://mlsanity.com/paper/2307.06771/) (2023)
- [Preference-Optimized Pareto Set Learning for Blackbox Optimization](https://mlsanity.com/paper/2408.09976/) (2024)
- [A Scalable Training Strategy for Blind Multi-Distribution Noise Removal](https://mlsanity.com/paper/2310.20064/) (2023)
- [D'OH: Decoder-Only Random Hypernetworks for Implicit Neural Representations](https://mlsanity.com/paper/2403.19163/) (2024)
- [Oracle Supervision Transfers for Hyperparameter Prediction in Model-Based Image Denoising](https://mlsanity.com/paper/2605.20479/) (2026)
- [One Evaluation, Any Operating Point: Hypernetwork-Amortized MeanFlow for 3D MRI Reconstruction](https://mlsanity.com/paper/2609.31655/) (2026)

## Abstract (arXiv)

Deep learning based techniques achieve state-of-the-art results in a wide range of image reconstruction tasks like compressed sensing. These methods almost always have hyperparameters, such as the weight coefficients that balance the different terms in the optimized loss function. The typical approach is to train the model for a hyperparameter setting determined with some empirical or theoretical justification. Thus, at inference time, the model can only compute reconstructions corresponding to the pre-determined hyperparameter values. In this work, we present a hypernetwork-based approach, called HyperRecon, to train reconstruction models that are agnostic to hyperparameter settings. At inference time, HyperRecon can efficiently produce diverse reconstructions, which would each correspond to different hyperparameter values. In this framework, the user is empowered to select the most useful output(s) based on their own judgement. We demonstrate our method in compressed sensing, super-resolution and denoising tasks, using two large-scale and publicly-available MRI datasets. Our code is available at https://github.com/alanqrwang/hyperrecon.