# Zero-shot super-resolution with a physically-motivated downsampling kernel for endomicroscopy

arXiv: https://arxiv.org/abs/2103.14015 · HTML: https://mlsanity.com/paper/2103.14015/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 29 references that are in the map.
- First posted on arXiv 25 Mar 2021.

## Builds on (most cited first, 15 of 15)

- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising](https://mlsanity.com/paper/1608.03981/) (2016)
- [Deep Learning for Image Super-resolution: A Survey](https://mlsanity.com/paper/1902.06068/) (2019)
- ["Zero-Shot" Super-Resolution using Deep Internal Learning](https://mlsanity.com/paper/1712.06087/) (2017)
- [Generative Adversarial Network in Medical Imaging: A Review](https://mlsanity.com/paper/1809.07294/) (2018)
- [Learning a Single Convolutional Super-Resolution Network for Multiple Degradations](https://mlsanity.com/paper/1712.06116/) (2017)
- [Gradient Magnitude Similarity Deviation: A Highly Efficient Perceptual Image Quality Index](https://mlsanity.com/paper/1308.3052/) (2013)
- [NTIRE 2020 Challenge on Real-World Image Super-Resolution: Methods and Results](https://mlsanity.com/paper/2005.01996/) (2020)
- [Seven ways to improve example-based single image super resolution](https://mlsanity.com/paper/1511.02228/) (2015)
- [Adversarial training with cycle consistency for unsupervised super-resolution in endomicroscopy](https://mlsanity.com/paper/1901.06988/) (2019)
- [Learning from Irregularly Sampled Data for Endomicroscopy Super-resolution: A Comparative Study of Sparse and Dense Approaches](https://mlsanity.com/paper/1911.13169/) (2019)
- [Effective deep learning training for single-image super-resolution in endomicroscopy exploiting video-registration-based reconstruction](https://mlsanity.com/paper/1803.08840/) (2018)
- [Can Deep Learning Relax Endomicroscopy Hardware Miniaturization Requirements?](https://mlsanity.com/paper/1806.08338/) (2018)
- [Image Super Resolution via Bilinear Pooling: Application to Confocal Endomicroscopy](https://mlsanity.com/paper/1906.07802/) (2019)
- [WhiteNNer-Blind Image Denoising via Noise Whiteness Priors](https://mlsanity.com/paper/1908.03238/) (2019)

## Built on by (most cited first, 1 of 1)

- [Hierarchical Perception Adversarial Learning Framework for Compressed Sensing MRI](https://mlsanity.com/paper/2302.10309/) (2023)

## Abstract (arXiv)

Super-resolution (SR) methods have seen significant advances thanks to the development of convolutional neural networks (CNNs). CNNs have been successfully employed to improve the quality of endomicroscopy imaging. Yet, the inherent limitation of research on SR in endomicroscopy remains the lack of ground truth high-resolution (HR) images, commonly used for both supervised training and reference-based image quality assessment (IQA). Therefore, alternative methods, such as unsupervised SR are being explored. To address the need for non-reference image quality improvement, we designed a novel zero-shot super-resolution (ZSSR) approach that relies only on the endomicroscopy data to be processed in a self-supervised manner without the need for ground-truth HR images. We tailored the proposed pipeline to the idiosyncrasies of endomicroscopy by introducing both: a physically-motivated Voronoi downscaling kernel accounting for the endomicroscope's irregular fibre-based sampling pattern, and realistic noise patterns. We also took advantage of video sequences to exploit a sequence of images for self-supervised zero-shot image quality improvement. We run ablation studies to assess our contribution in regards to the downscaling kernel and noise simulation. We validate our methodology on both synthetic and original data. Synthetic experiments were assessed with reference-based IQA, while our results for original images were evaluated in a user study conducted with both expert and non-expert observers. The results demonstrated superior performance in image quality of ZSSR reconstructions in comparison to the baseline method. The ZSSR is also competitive when compared to supervised single-image SR, especially being the preferred reconstruction technique by experts.