# Image Completion with Heterogeneously Filtered Spectral Hints

arXiv: https://arxiv.org/abs/2211.03700 · HTML: https://mlsanity.com/paper/2211.03700/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 67 references that are in the map.
- First posted on arXiv 7 Nov 2022.

## Builds on (most cited first, 30 of 36)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [Describing Textures in the Wild](https://mlsanity.com/paper/1311.3618/) (2013)
- [On the Spectral Bias of Neural Networks](https://mlsanity.com/paper/1806.08734/) (2018)
- [Training Generative Adversarial Networks with Limited Data](https://mlsanity.com/paper/2006.06676/) (2020)
- [Alias-Free Generative Adversarial Networks](https://mlsanity.com/paper/2106.12423/) (2021)
- [StyleCLIP: Text-Driven Manipulation of StyleGAN Imagery](https://mlsanity.com/paper/2103.17249/) (2021)
- [Resolution-robust Large Mask Inpainting with Fourier Convolutions](https://mlsanity.com/paper/2109.07161/) (2021)
- [Interpreting the Latent Space of GANs for Semantic Face Editing](https://mlsanity.com/paper/1907.10786/) (2019)
- [GANSpace: Discovering Interpretable GAN Controls](https://mlsanity.com/paper/2004.02546/) (2020)
- [Free-Form Image Inpainting with Gated Convolution](https://mlsanity.com/paper/1806.03589/) (2018)
- [Generative Image Inpainting with Contextual Attention](https://mlsanity.com/paper/1801.07892/) (2018)
- [Image Inpainting for Irregular Holes Using Partial Convolutions](https://mlsanity.com/paper/1804.07723/) (2018)
- [StyleFlow: Attribute-conditioned Exploration of StyleGAN-Generated Images using Conditional Continuous Normalizing Flows](https://mlsanity.com/paper/2008.02401/) (2020)
- [StyleRig: Rigging StyleGAN for 3D Control over Portrait Images](https://mlsanity.com/paper/2004.00121/) (2020)
- [Exploiting Deep Generative Prior for Versatile Image Restoration and Manipulation](https://mlsanity.com/paper/2003.13659/) (2020)
- [Large Scale Image Completion via Co-Modulated Generative Adversarial Networks](https://mlsanity.com/paper/2103.10428/) (2021)
- [Photorealistic Style Transfer via Wavelet Transforms](https://mlsanity.com/paper/1903.09760/) (2019)
- [Contextual Residual Aggregation for Ultra High-Resolution Image Inpainting](https://mlsanity.com/paper/2005.09704/) (2020)
- [Aggregated Contextual Transformations for High-Resolution Image Inpainting](https://mlsanity.com/paper/2104.01431/) (2021)
- [Spectral Representations for Convolutional Neural Networks](https://mlsanity.com/paper/1506.03767/) (2015)

## Built on by (most cited first, 17 of 17)

- [Versatile Diffusion: Text, Images and Variations All in One Diffusion Model](https://mlsanity.com/paper/2211.08332/) (2022)
- [HD-Painter: High-Resolution and Prompt-Faithful Text-Guided Image Inpainting with Diffusion Models](https://mlsanity.com/paper/2312.14091/) (2023)
- [Smooth Diffusion: Crafting Smooth Latent Spaces in Diffusion Models](https://mlsanity.com/paper/2312.04410/) (2023)
- [Zero-shot Generative Model Adaptation via Image-specific Prompt Learning](https://mlsanity.com/paper/2304.03119/) (2023)
- [Towards Enhanced Image Inpainting: Mitigating Unwanted Object Insertion and Preserving Color Consistency](https://mlsanity.com/paper/2312.04831/) (2023)
- [Zero-Painter: Training-Free Layout Control for Text-to-Image Synthesis](https://mlsanity.com/paper/2406.04032/) (2024)
- [PixelHacker: Image Inpainting with Structural and Semantic Consistency](https://mlsanity.com/paper/2504.20438/) (2025)
- [Automatic High Resolution Wire Segmentation and Removal](https://mlsanity.com/paper/2304.00221/) (2023)
- [SmartFreeEdit: Mask-Free Spatial-Aware Image Editing with Complex Instruction Understanding](https://mlsanity.com/paper/2504.12704/) (2025)
- [DMAT: A Dynamic Mask-Aware Transformer for Human De-occlusion](https://mlsanity.com/paper/2402.04558/) (2024)
- [PainterNet: Adaptive Image Inpainting with Actual-Token Attention and Diverse Mask Control](https://mlsanity.com/paper/2412.01223/) (2024)
- [Binary Diffusion Probabilistic Model](https://mlsanity.com/paper/2501.13915/) (2025)
- [Towards Seamless Borders: A Method for Mitigating Inconsistencies in Image Inpainting and Outpainting](https://mlsanity.com/paper/2506.12530/) (2025)
- [Windowed-FourierMixer: Enhancing Clutter-Free Room Modeling with Fourier Transform](https://mlsanity.com/paper/2402.18287/) (2024)
- [RETHINED: A New Benchmark and Baseline for Real-Time High-Resolution Image Inpainting On Edge Devices](https://mlsanity.com/paper/2503.14757/) (2025)
- [MILD: Multi-Layer Diffusion Strategy for Complex and Precise Multi-IP Aware Human Erasing](https://mlsanity.com/paper/2508.06543/) (2025)
- [Aligned Stable Inpainting: Mitigating Unwanted Object Insertion and Preserving Color Consistency](https://mlsanity.com/paper/2601.15368/) (2026)

## Abstract (arXiv)

Image completion with large-scale free-form missing regions is one of the most challenging tasks for the computer vision community. While researchers pursue better solutions, drawbacks such as pattern unawareness, blurry textures, and structure distortion remain noticeable, and thus leave space for improvement. To overcome these challenges, we propose a new StyleGAN-based image completion network, Spectral Hint GAN (SH-GAN), inside which a carefully designed spectral processing module, Spectral Hint Unit, is introduced. We also propose two novel 2D spectral processing strategies, Heterogeneous Filtering and Gaussian Split that well-fit modern deep learning models and may further be extended to other tasks. From our inclusive experiments, we demonstrate that our model can reach FID scores of 3.4134 and 7.0277 on the benchmark datasets FFHQ and Places2, and therefore outperforms prior works and reaches a new state-of-the-art. We also prove the effectiveness of our design via ablation studies, from which one may notice that the aforementioned challenges, i.e. pattern unawareness, blurry textures, and structure distortion, can be noticeably resolved. Our code will be open-sourced at: https://github.com/SHI-Labs/SH-GAN.