# Contextual Attention Mechanism, SRGAN Based Inpainting System for Eliminating Interruptions from Images

arXiv: https://arxiv.org/abs/2204.02591 · HTML: https://mlsanity.com/paper/2204.02591/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 42 references that are in the map.
- First posted on arXiv 6 Apr 2022.

## Builds on (most cited first, 11 of 11)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Generative Image Inpainting with Contextual Attention](https://mlsanity.com/paper/1801.07892/) (2018)
- [EdgeConnect: Generative Image Inpainting with Adversarial Edge Learning](https://mlsanity.com/paper/1901.00212/) (2019)
- [Semantic Image Inpainting with Deep Generative Models](https://mlsanity.com/paper/1607.07539/) (2016)
- [Compatible and Diverse Fashion Image Inpainting](https://mlsanity.com/paper/1902.01096/) (2019)
- [Adversarial Inpainting of Medical Image Modalities](https://mlsanity.com/paper/1810.06621/) (2018)
- [Local Deep-Feature Alignment for Unsupervised Dimension Reduction](https://mlsanity.com/paper/1904.09747/) (2019)
- [Depth Image Inpainting: Improving Low Rank Matrix Completion with Low Gradient Regularization](https://mlsanity.com/paper/1604.05817/) (2016)
- [Multi-modal Face Pose Estimation with Multi-task Manifold Deep Learning](https://mlsanity.com/paper/1712.06467/) (2017)
- [Generative Image Inpainting with Submanifold Alignment](https://mlsanity.com/paper/1908.00211/) (2019)
- [Boosted GAN with Semantically Interpretable Information for Image Inpainting](https://mlsanity.com/paper/1908.04503/) (2019)

## Built on by (most cited first, 1 of 1)

- [SocraSynth: Multi-LLM Reasoning with Conditional Statistics](https://mlsanity.com/paper/2402.06634/) (2024)

## Abstract (arXiv)

The new alternative is to use deep learning to inpaint any image by utilizing image classification and computer vision techniques. In general, image inpainting is a task of recreating or reconstructing any broken image which could be a photograph or oil/acrylic painting. With the advancement in the field of Artificial Intelligence, this topic has become popular among AI enthusiasts. With our approach, we propose an initial end-to-end pipeline for inpainting images using a complete Machine Learning approach instead of a conventional application-based approach. We first use the YOLO model to automatically identify and localize the object we wish to remove from the image. Using the result obtained from the model we can generate a mask for the same. After this, we provide the masked image and original image to the GAN model which uses the Contextual Attention method to fill in the region. It consists of two generator networks and two discriminator networks and is also called a coarse-to-fine network structure. The two generators use fully convolutional networks while the global discriminator gets hold of the entire image as input while the local discriminator gets the grip of the filled region as input. The contextual Attention mechanism is proposed to effectively borrow the neighbor information from distant spatial locations for reconstructing the missing pixels. The third part of our implementation uses SRGAN to resolve the inpainted image back to its original size. Our work is inspired by the paper Free-Form Image Inpainting with Gated Convolution and Generative Image Inpainting with Contextual Attention.