mlsanityData to 14 Oct 2026

Free-Form Image Inpainting with Gated Convolution

arXiv 1806.03589 ↗First posted 10 Jun 2018cs.CV cs.GR cs.LGcited by 594

Not on the map. Why?

Cited by 72 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We present a generative image inpainting system to complete images with free-form mask and guidance. The system is based on gated convolutions learned from millions of images without additional labelling efforts. The proposed gated convolution solves the issue of vanilla convolution that treats all input pixels as valid ones, generalizes partial convolution by providing a learnable dynamic feature selection mechanism for each channel at each spatial location across all layers. Moreover, as free-form masks may appear anywhere in images with any shape, global and local GANs designed for a single rectangular mask are not applicable. Thus, we also present a patch-based GAN loss, named SN-PatchGAN, by applying spectral-normalized discriminator on dense image patches. SN-PatchGAN is simple in formulation, fast and stable in training. Results on automatic image inpainting and user-guided extension demonstrate that our system generates higher-quality and more flexible results than previous methods. Our system helps user quickly remove distracting objects, modify image layouts, clear watermarks and edit faces. Code, demo and models are available at: https://github.com/JiahuiYu/generative_inpainting

Key facts. Cited by 594 papers in the map (data through 14 Oct 2026). First posted on arXiv 10 Jun 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 131; 2022: 139; 2023: 124; 2024: 99; 2025: 70; 2026: 31.

594 papers.

Sort:

By area:

  1. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  2. 2022RePaint: Inpainting using Denoising Diffusion Probabilistic Modelscited by 1,430
  3. 2021Palette: Image-to-Image Diffusion Modelscited by 1,300
  4. 2021SimMIM: A Simple Framework for Masked Image Modelingcited by 1,153
  5. 2022MaskGIT: Masked Generative Image Transformercited by 1,099
  6. 2021Resolution-robust Large Mask Inpainting with Fourier Convolutionscited by 738
  7. 2021ILVR: Conditioning Method for Denoising Diffusion Probabilistic Modelscited by 635
  8. 2023Make-It-3D: High-Fidelity 3D Creation from A Single Image with Diffusion Priorcited by 343
  9. 2021Come-Closer-Diffuse-Faster: Accelerating Conditional Diffusion Models for Inverse Problems through Stochastic Contractioncited by 307
  10. 2023DiffIR: Efficient Diffusion Model for Image Restorationcited by 255
  11. 2022Imagen Editor and EditBench: Advancing and Evaluating Text-Guided Image Inpaintingcited by 233
  12. 2023TOPIQ: A Top-down Approach from Semantics to Distortions for Image Quality Assessmentcited by 226
38 more
  1. 2021Large Scale Image Completion via Co-Modulated Generative Adversarial Networkscited by 220
  2. 2022MAT: Mask-Aware Transformer for Large Hole Image Inpaintingcited by 218
  3. 2023I2SB: Image-to-Image Schr"odinger Bridgecited by 218
  4. 2023A Task is Worth One Word: Learning with Task Prompts for High-Quality Versatile Image Inpaintingcited by 169
  5. 2023Flow Matching in Latent Spacecited by 160
  6. 2023Solving Linear Inverse Problems Provably via Posterior Sampling with Latent Diffusion Modelscited by 157
  7. 2022NeuralLift-360: Lifting An In-the-wild 2D Photo to A 3D Object with 360deg Viewscited by 148
  8. 2021High-Fidelity Pluralistic Image Completion with Transformerscited by 134
  9. 2023Score-Based Diffusion Models as Principled Priors for Inverse Imagingcited by 127
  10. 2022Towards An End-to-End Framework for Flow-Guided Video Inpaintingcited by 113
  11. 2023Ambient Diffusion: Learning Clean Distributions from Corrupted Datacited by 111
  12. 2021Generating Diverse Structure for Image Inpainting With Hierarchical VQ-VAEcited by 106
  13. 2022DifFace: Blind Face Restoration with Diffused Error Contractioncited by 97
  14. 2021FuseFormer: Fusing Fine-Grained Information in Transformers for Video Inpaintingcited by 95
  15. 2021Domain Impression: A Source Data Free Domain Adaptation Methodcited by 91
  16. 2024VmambaIR: Visual State Space Model for Image Restorationcited by 90
  17. 2021Aggregated Contextual Transformations for High-Resolution Image Inpaintingcited by 89
  18. 2021Equivariant Imaging: Learning Beyond the Range Spacecited by 83
  19. 2022Incremental Transformer Structure Enhanced Image Inpainting with Masking Positional Encodingcited by 83
  20. 2022InfiniteNature-Zero: Learning Perpetual View Generation of Natural Scenes from Single Imagescited by 82
  21. 2023AVID: Any-Length Video Inpainting with Diffusion Modelcited by 82
  22. 2021Diverse Image Inpainting with Bidirectional and Autoregressive Transformerscited by 80
  23. 2021PD-GAN: Probabilistic Diverse GAN for Image Inpaintingcited by 80
  24. 2021GeoSim: Realistic Video Simulation via Geometry-Aware Composition for Self-Drivingcited by 75
  25. 2024VRP-SAM: SAM with Visual Reference Promptcited by 75
  26. 2021Open-World Entity Segmentationcited by 74
  27. 2021Intermediate Layer Optimization for Inverse Problems using Deep Generative Modelscited by 70
  28. 2023HD-Painter: High-Resolution and Prompt-Faithful Text-Guided Image Inpainting with Diffusion Modelscited by 67
  29. 2021Image Inpainting via Conditional Texture and Structure Dual Generationcited by 66
  30. 2023Inst-Inpaint: Instructing to Remove Objects with Diffusion Modelscited by 66
  31. 2022High-fidelity 3D GAN Inversion by Pseudo-multi-view Optimizationcited by 65
  32. 2023Single Motion Diffusioncited by 65
  33. 2024Zero-shot Image Editing with Reference Imitationcited by 65
  34. 2024Efficient Diffusion Model for Image Restoration by Residual Shiftingcited by 64
  35. 2021Making Images Real Again: A Comprehensive Survey on Deep Image Compositioncited by 63
  36. 2021PISE: Person Image Synthesis and Editing with Decoupled GANcited by 62
  37. 2021InfinityGAN: Towards Infinite-Pixel Image Synthesiscited by 61
  38. 2023Uni-paint: A Unified Framework for Multimodal Image Inpainting with Pretrained Diffusion Modelcited by 60

50 most cited of 594 All 594 (JSON)