mlsanityData to 14 Oct 2026

Reference Guided Image Inpainting using Facial Attributes

arXiv 2301.08044 ↗First posted 19 Jan 2023cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Image inpainting is a technique of completing missing pixels such as occluded region restoration, distracting objects removal, and facial completion. Among these inpainting tasks, facial completion algorithm performs face inpainting according to the user direction. Existing approaches require delicate and well controlled input by the user, thus it is difficult for an average user to provide the guidance sufficiently accurate for the algorithm to generate desired results. To overcome this limitation, we propose an alternative user-guided inpainting architecture that manipulates facial attributes using a single reference image as the guide. Our end-to-end model consists of attribute extractors for accurate reference image attribute transfer and an inpainting model to map the attributes realistically and accurately to generated images. We customize MS-SSIM loss and learnable bidirectional attention maps in which importance structures remain intact even with irregular shaped masks. Based on our evaluation using the publicly available dataset CelebA-HQ, we demonstrate that the proposed method delivers superior performance compared to some state-of-the-art methods specialized in inpainting tasks.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 27 of its 37 references that are in the map. First posted on arXiv 19 Jan 2023.

Builds on

27 of 37 references.

References · 27

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  4. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  5. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  6. 2022Generative Adversarial Networkscited by 5,627
  7. 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,988
  8. 2017Improved Training of Wasserstein GANscited by 2,235
  9. 2018Spectral Normalization for Generative Adversarial Networkscited by 1,636
  10. 2016Context Encoders: Feature Learning by Inpaintingcited by 1,428
  11. 2017StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translationcited by 1,039
  12. 2018Free-Form Image Inpainting with Gated Convolutioncited by 594
15 more
  1. 2018Generative Image Inpainting with Contextual Attentioncited by 580
  2. 2018Image Inpainting for Irregular Holes Using Partial Convolutionscited by 505
  3. 2017AttGAN: Facial Attribute Editing by Only Changing What You Wantcited by 242
  4. 2016EnhanceNet: Single Image Super-Resolution Through Automated Texture Synthesiscited by 178
  5. 2019Pluralistic Image Completioncited by 162
  6. 2019STGAN: A Unified Selective Transfer Network for Arbitrary Image Attribute Editingcited by 120
  7. 2019SC-FEGAN: Face Editing Generative Adversarial Network with User's Sketch and Colorcited by 106
  8. 2021PD-GAN: Probabilistic Diverse GAN for Image Inpaintingcited by 80
  9. 2019Image Inpainting with Learnable Bidirectional Attention Mapscited by 64
  10. 2018Partial Convolution based Paddingcited by 29
  11. 2020CAFE-GAN: Arbitrary Face Attribute Editing with Complementary Attention Featurecited by 20
  12. 2019PiiGAN: Generative Adversarial Networks for Pluralistic Image Inpaintingcited by 17
  13. 2020PA-GAN: Progressive Attention Generative Adversarial Network for Facial Attribute Editingcited by 13
  14. 2018Semi-parametric Image Inpaintingcited by 6
  15. 2021Extreme Face Inpainting with Sketch-Guided Conditional GANcited by 1

Built on by

Not cited yet.