# Spherical Image Inpainting with Frame Transformation and Data-driven Prior Deep Networks

arXiv: https://arxiv.org/abs/2209.14604 · HTML: https://mlsanity.com/paper/2209.14604/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 38 references that are in the map.
- First posted on arXiv 29 Sep 2022.

## Builds on (most cited first, 16 of 16)

- [Matterport3D: Learning from RGB-D Data in Indoor Environments](https://mlsanity.com/paper/1709.06158/) (2017)
- [Plug-and-Play Image Restoration with Deep Denoiser Prior](https://mlsanity.com/paper/2008.13751/) (2020)
- [Learning Deep CNN Denoiser Prior for Image Restoration](https://mlsanity.com/paper/1704.03264/) (2017)
- [Plug-and-Play ADMM for Image Restoration: Fixed Point Convergence and Applications](https://mlsanity.com/paper/1605.01710/) (2016)
- [Multi-level Wavelet-CNN for Image Restoration](https://mlsanity.com/paper/1805.07071/) (2018)
- [DoubleU-Net: A Deep Convolutional Neural Network for Medical Image Segmentation](https://mlsanity.com/paper/2006.04868/) (2020)
- [Tuning-free Plug-and-Play Proximal Algorithm for Inverse Imaging Problems](https://mlsanity.com/paper/2002.09611/) (2020)
- [SphereSR: 360deg Image Super-Resolution with Arbitrary Projection via Continuous Spherical Image Representation](https://mlsanity.com/paper/2112.06536/) (2021)
- [FBSNet: A Fast Bilateral Symmetrical Network for Real-Time Semantic Segmentation](https://mlsanity.com/paper/2109.00699/) (2021)
- [Deep Plug-and-play Prior for Low-rank Tensor Completion](https://mlsanity.com/paper/1905.04449/) (2019)
- [Snow Mask Guided Adaptive Residual Network for Image Snow Removal](https://mlsanity.com/paper/2207.04754/) (2022)
- [A Fixation-based 360deg Benchmark Dataset for Salient Object Detection](https://mlsanity.com/paper/2001.07960/) (2020)
- [Image Restoration and Reconstruction using Variable Splitting and Class-adapted Image Priors](https://mlsanity.com/paper/1602.04052/) (2016)
- [Convolutional Neural Networks for Spherical Signal Processing via Spherical Haar Tight Framelets](https://mlsanity.com/paper/2201.07890/) (2022)
- [Multiple Degradation and Reconstruction Network for Single Image Denoising via Knowledge Distillation](https://mlsanity.com/paper/2204.13873/) (2022)
- [Overlapping Domain Decomposition Methods for Ptychographic Imaging](https://mlsanity.com/paper/2011.00162/) (2020)

## Built on by (most cited first, 1 of 1)

- [Revisiting Nonlocal Self-Similarity from Continuous Representation](https://mlsanity.com/paper/2401.00708/) (2024)

## Abstract (arXiv)

Spherical image processing has been widely applied in many important fields, such as omnidirectional vision for autonomous cars, global climate modelling, and medical imaging. It is non-trivial to extend an algorithm developed for flat images to the spherical ones. In this work, we focus on the challenging task of spherical image inpainting with deep learning-based regularizer. Instead of a naive application of existing models for planar images, we employ a fast directional spherical Haar framelet transform and develop a novel optimization framework based on a sparsity assumption of the framelet transform. Furthermore, by employing progressive encoder-decoder architecture, a new and better-performed deep CNN denoiser is carefully designed and works as an implicit regularizer. Finally, we use a plug-and-play method to handle the proposed optimization model, which can be implemented efficiently by training the CNN denoiser prior. Numerical experiments are conducted and show that the proposed algorithms can greatly recover damaged spherical images and achieve the best performance over purely using deep learning denoiser and plug-and-play model.