# MISF: Multi-level Interactive Siamese Filtering for High-Fidelity Image Inpainting

arXiv: https://arxiv.org/abs/2203.06304 · HTML: https://mlsanity.com/paper/2203.06304/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 35 references that are in the map.
- First posted on arXiv 12 Mar 2022.

## Builds on (most cited first, 24 of 24)

- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [Conditional Generative Adversarial Nets](https://mlsanity.com/paper/1411.1784/) (2014)
- [Context Encoders: Feature Learning by Inpainting](https://mlsanity.com/paper/1604.07379/) (2016)
- [Free-Form Image Inpainting with Gated Convolution](https://mlsanity.com/paper/1806.03589/) (2018)
- [Generative Image Inpainting with Contextual Attention](https://mlsanity.com/paper/1801.07892/) (2018)
- [Image Inpainting for Irregular Holes Using Partial Convolutions](https://mlsanity.com/paper/1804.07723/) (2018)
- [Dynamic Convolution: Attention over Convolution Kernels](https://mlsanity.com/paper/1912.03458/) (2019)
- [EnhanceNet: Single Image Super-Resolution Through Automated Texture Synthesis](https://mlsanity.com/paper/1612.07919/) (2016)
- [Semantic Image Inpainting with Deep Generative Models](https://mlsanity.com/paper/1607.07539/) (2016)
- [Burst Denoising with Kernel Prediction Networks](https://mlsanity.com/paper/1712.02327/) (2017)
- [StructureFlow: Image Inpainting via Structure-aware Appearance Flow](https://mlsanity.com/paper/1908.03852/) (2019)
- [Recurrent Feature Reasoning for Image Inpainting](https://mlsanity.com/paper/2008.03737/) (2020)
- [Shift-Net: Image Inpainting via Deep Feature Rearrangement](https://mlsanity.com/paper/1801.09392/) (2018)
- [Auto-Exposure Fusion for Single-Image Shadow Removal](https://mlsanity.com/paper/2103.01255/) (2021)
- [Image Inpainting via Conditional Texture and Structure Dual Generation](https://mlsanity.com/paper/2108.09760/) (2021)
- [EfficientDeRain: Learning Pixel-wise Dilation Filtering for High-Efficiency Single-Image Deraining](https://mlsanity.com/paper/2009.09238/) (2020)
- [Learning to Synthesize Motion Blur](https://mlsanity.com/paper/1811.11745/) (2018)
- [Geometry-Aware Face Completion and Editing](https://mlsanity.com/paper/1809.02967/) (2018)
- [JPGNet: Joint Predictive Filtering and Generative Network for Image Inpainting](https://mlsanity.com/paper/2107.04281/) (2021)
- [Dunhuang Grottoes Painting Dataset and Benchmark](https://mlsanity.com/paper/1907.04589/) (2019)
- [Efficient Algorithms for Densest Subgraph Discovery](https://mlsanity.com/paper/1906.00341/) (2019)
- [Sparta: Spatially Attentive and Adversarially Robust Activation](https://mlsanity.com/paper/2105.08269/) (2021)

## Built on by (most cited first, 30 of 35)

- [Leveraging Inpainting for Single-Image Shadow Removal](https://mlsanity.com/paper/2302.05361/) (2023)
- [SmartEraser: Remove Anything from Images using Masked-Region Guidance](https://mlsanity.com/paper/2501.08279/) (2025)
- [Bidirectional Autoregressive Diffusion Model for Dance Generation](https://mlsanity.com/paper/2402.04356/) (2024)
- [HINT: High-quality INPainting Transformer with Mask-Aware Encoding and Enhanced Attention](https://mlsanity.com/paper/2402.14185/) (2024)
- [Coarse-to-fine Task-driven Inpainting for Geoscience Images](https://mlsanity.com/paper/2211.11059/) (2022)
- [Beyond the Field-of-View: Enhancing Scene Visibility and Perception with Clip-Recurrent Transformer](https://mlsanity.com/paper/2211.11293/) (2022)
- [Deficiency-Aware Masked Transformer for Video Inpainting](https://mlsanity.com/paper/2307.08629/) (2023)
- [Decoupled Residual Denoising Diffusion Models for Unified and Data Efficient Image-to-Image Translation](https://mlsanity.com/paper/2606.01048/) (2026)
- [Learning Global-aware Kernel for Image Harmonization](https://mlsanity.com/paper/2305.11676/) (2023)
- [BVINet: Unlocking Blind Video Inpainting with Zero Annotations](https://mlsanity.com/paper/2502.01181/) (2025)
- [Structure-Informed Shadow Removal Networks](https://mlsanity.com/paper/2301.03182/) (2023)
- [SuperInpaint: Learning Detail-Enhanced Attentional Implicit Representation for Super-resolutional Image Inpainting](https://mlsanity.com/paper/2307.14489/) (2023)
- [Seeing Behind Dynamic Occlusions with Event Cameras](https://mlsanity.com/paper/2307.15829/) (2023)
- [Hierarchical Contrastive Learning for Pattern-Generalizable Image Corruption Detection](https://mlsanity.com/paper/2308.14061/) (2023)
- [SAIR: Learning Semantic-aware Implicit Representation](https://mlsanity.com/paper/2310.09285/) (2023)
- [CMAMRNet: A Contextual Mask-Aware Network Enhancing Mural Restoration Through Comprehensive Mask Guidance](https://mlsanity.com/paper/2508.07140/) (2025)
- [Learning Restoration is Not Enough: Transfering Identical Mapping for Single-Image Shadow Removal](https://mlsanity.com/paper/2305.10640/) (2023)
- [Line Drawing Guided Progressive Inpainting of Mural Damage](https://mlsanity.com/paper/2211.06649/) (2022)
- [A Structure-Guided Diffusion Model for Large-Hole Image Completion](https://mlsanity.com/paper/2211.10437/) (2022)
- [Gorgeous: Create Your Desired Character Facial Makeup from Any Ideas](https://mlsanity.com/paper/2404.13944/) (2024)
- [MxT: Mamba x Transformer for Image Inpainting](https://mlsanity.com/paper/2407.16126/) (2024)
- [Leveraging Adaptive Implicit Representation Mapping for Ultra High-Resolution Image Segmentation](https://mlsanity.com/paper/2407.21256/) (2024)
- [FCDM: A Physics-Guided Bidirectional Frequency Aware Convolution and Diffusion-Based Model for Sinogram Inpainting](https://mlsanity.com/paper/2409.06714/) (2024)
- [Frequency-Domain Fusion Transformer for Image Inpainting](https://mlsanity.com/paper/2506.18437/) (2025)
- [Image Completion via Dual-path Cooperative Filtering](https://mlsanity.com/paper/2305.00379/) (2023)
- [AURA : Automatic Mask Generator using Randomized Input Sampling for Object Removal](https://mlsanity.com/paper/2305.07857/) (2023)
- [One-to-many Reconstruction of 3D Geometry of cultural Artifacts using a synthetically trained Generative Model](https://mlsanity.com/paper/2402.08310/) (2024)
- [CLII: Visual-Text Inpainting via Cross-Modal Predictive Interaction](https://mlsanity.com/paper/2407.16204/) (2024)
- [SEM-Net: Efficient Pixel Modelling for image inpainting with Spatially Enhanced SSM](https://mlsanity.com/paper/2411.06318/) (2024)
- [High-Fidelity Image Inpainting with Multimodal Guided GAN Inversion](https://mlsanity.com/paper/2504.12844/) (2025)

## Abstract (arXiv)

Although achieving significant progress, existing deep generative inpainting methods are far from real-world applications due to the low generalization across different scenes. As a result, the generated images usually contain artifacts or the filled pixels differ greatly from the ground truth. Image-level predictive filtering is a widely used image restoration technique, predicting suitable kernels adaptively according to different input scenes. Inspired by this inherent advantage, we explore the possibility of addressing image inpainting as a filtering task. To this end, we first study the advantages and challenges of image-level predictive filtering for image inpainting: the method can preserve local structures and avoid artifacts but fails to fill large missing areas. Then, we propose semantic filtering by conducting filtering on the deep feature level, which fills the missing semantic information but fails to recover the details. To address the issues while adopting the respective advantages, we propose a novel filtering technique, i.e., Multilevel Interactive Siamese Filtering (MISF), which contains two branches: kernel prediction branch (KPB) and semantic & image filtering branch (SIFB). These two branches are interactively linked: SIFB provides multi-level features for KPB while KPB predicts dynamic kernels for SIFB. As a result, the final method takes the advantage of effective semantic & image-level filling for high-fidelity inpainting. We validate our method on three challenging datasets, i.e., Dunhuang, Places2, and CelebA. Our method outperforms state-of-the-art baselines on four metrics, i.e., L1, PSNR, SSIM, and LPIPS. Please try the released code and model at https://github.com/tsingqguo/misf.