# Efficient Neural Generation of 4K Masks for Homogeneous Diffusion Inpainting

arXiv: https://arxiv.org/abs/2303.10096 · HTML: https://mlsanity.com/paper/2303.10096/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 27 references that are in the map.
- First posted on arXiv 17 Mar 2023.

## Builds on (most cited first, 10 of 10)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [iPiano: Inertial Proximal Algorithm for Non-Convex Optimization](https://mlsanity.com/paper/1404.4805/) (2014)
- [Adaptive Image Sampling using Deep Learning and its Application on X-Ray Fluorescence Image Reconstruction](https://mlsanity.com/paper/1812.10836/) (2018)
- [Learning Sparse Masks for Diffusion-based Image Inpainting](https://mlsanity.com/paper/2110.02636/) (2021)
- [Efficient Data Optimisation for Harmonic Inpainting with Finite Elements](https://mlsanity.com/paper/2105.01586/) (2021)
- [Image Inpainting Using Wasserstein Generative Adversarial Imputation Network](https://mlsanity.com/paper/2106.15341/) (2021)
- [Sparse Inpainting with Smoothed Particle Hydrodynamics](https://mlsanity.com/paper/2011.11289/) (2020)
- [Deep Spatial and Tonal Data Optimisation for Homogeneous Diffusion Inpainting](https://mlsanity.com/paper/2208.14371/) (2022)
- [Domain Decomposition Algorithms for Real-time Homogeneous Diffusion Inpainting in 4K](https://mlsanity.com/paper/2110.03946/) (2021)
- [A Wasserstein GAN for Joint Learning of Inpainting and Spatial Optimisation](https://mlsanity.com/paper/2202.05623/) (2022)

## Built on by (most cited first, 2 of 2)

- [Deep Learning-based Image and Video Inpainting: A Survey](https://mlsanity.com/paper/2401.03395/) (2024)
- [Advancements and Trends in Ultra-High-Resolution Image Processing: An Overview](https://mlsanity.com/paper/2312.00250/) (2023)

## Abstract (arXiv)

With well-selected data, homogeneous diffusion inpainting can reconstruct images from sparse data with high quality. While 4K colour images of size 3840 x 2160 can already be inpainted in real time, optimising the known data for applications like image compression remains challenging: Widely used stochastic strategies can take days for a single 4K image. Recently, a first neural approach for this so-called mask optimisation problem offered high speed and good quality for small images. It trains a mask generation network with the help of a neural inpainting surrogate. However, these mask networks can only output masks for the resolution and mask density they were trained for. We solve these problems and enable mask optimisation for high-resolution images through a neuroexplicit coarse-to-fine strategy. Additionally, we improve the training and interpretability of mask networks by including a numerical inpainting solver directly into the network. This allows to generate masks for 4K images in around 0.6 seconds while exceeding the quality of stochastic methods on practically relevant densities. Compared to popular existing approaches, this is an acceleration of up to four orders of magnitude.