mlsanityData to 7 Oct 2026

Learning a Sketch Tensor Space for Image Inpainting of Man-made Scenes

arXiv 2103.15087 ↗First posted 28 Mar 2021cs.CVcited by 25

Not on the map. Why?

Abstract (arXiv)

This paper studies the task of inpainting man-made scenes. It is very challenging due to the difficulty in preserving the visual patterns of images, such as edges, lines, and junctions. Especially, most previous works are failed to restore the object/building structures for images of man-made scenes. To this end, this paper proposes learning a Sketch Tensor (ST) space for inpainting man-made scenes. Such a space is learned to restore the edges, lines, and junctions in images, and thus makes reliable predictions of the holistic image structures. To facilitate the structure refinement, we propose a Multi-scale Sketch Tensor inpainting (MST) network, with a novel encoder-decoder structure. The encoder extracts lines and edges from the input images to project them into an ST space. From this space, the decoder is learned to restore the input images. Extensive experiments validate the efficacy of our model. Furthermore, our model can also achieve competitive performance in inpainting general nature images over the competitors.

Key facts. Cited by 25 papers in the map (data through 7 Oct 2026). Builds on 31 of its 52 references that are in the map. First posted on arXiv 28 Mar 2021.

Builds on

31 of 52 references.

References · 31

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  3. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  4. 2016Perceptual Losses for Real-Time Style Transfer and Super-Resolutioncited by 2,678
  5. 2018Spectral Normalization for Generative Adversarial Networkscited by 1,618
  6. 2019Class-Balanced Loss Based on Effective Number of Samplescited by 1,239
  7. 2017High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANscited by 1,176
  8. 2015Multi-Scale Context Aggregation by Dilated Convolutionscited by 1,128
  9. 2019MaskGAN: Towards Diverse and Interactive Facial Image Manipulationcited by 624
  10. 2018Free-Form Image Inpainting with Gated Convolutioncited by 594
  11. 2018Generative Image Inpainting with Contextual Attentioncited by 578
  12. 2018Image Inpainting for Irregular Holes Using Partial Convolutionscited by 505
19 more
  1. 2018Efficient Attention: Attention with Linear Complexitiescited by 366
  2. 2020Contextual Residual Aggregation for Ultra High-Resolution Image Inpaintingcited by 124
  3. 2017Real-Time User-Guided Image Colorization with Learned Deep Priorscited by 123
  4. 2019StructureFlow: Image Inpainting via Structure-aware Appearance Flowcited by 118
  5. 2020Recurrent Feature Reasoning for Image Inpaintingcited by 107
  6. 2019SC-FEGAN: Face Editing Generative Adversarial Network with User's Sketch and Colorcited by 105
  7. 2020Rethinking Image Inpainting via a Mutual Encoder-Decoder with Feature Equalizationscited by 102
  8. 2019Coherent Semantic Attention for Image Inpaintingcited by 97
  9. 2019Learning Pyramid-Context Encoder Network for High-Quality Image Inpaintingcited by 94
  10. 2019Foreground-aware Image Inpaintingcited by 90
  11. 2019Image inpainting: A reviewcited by 89
  12. 2020High-Resolution Image Inpainting with Iterative Confidence Feedback and Guided Upsamplingcited by 79
  13. 2016Scribbler: Controlling Deep Image Synthesis with Sketch and Colorcited by 77
  14. 2019End-to-End Wireframe Parsingcited by 72
  15. 2020Learning to Parse Wireframes in Images of Man-Made Environmentscited by 71
  16. 2020Holistically-Attracted Wireframe Parsingcited by 63
  17. 2018SPG-Net: Segmentation Prediction and Guidance Network for Image Inpaintingcited by 58
  18. 2020Guidance and Evaluation: Semantic-Aware Image Inpainting for Mixed Scenescited by 43
  19. 2020Learning to Incorporate Structure Knowledge for Image Inpaintingcited by 37

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 2; 2022: 11; 2023: 4; 2024: 6; 2025: 1; 2026: 1.

25 papers.

Sort:

By area:

  1. 2022Incremental Transformer Structure Enhanced Image Inpainting with Masking Positional Encodingcited by 83
  2. 2024Deep Learning-based Image and Video Inpainting: A Surveycited by 44
  3. 2022NeRF-In: Free-Form NeRF Inpainting with RGB-D Priorscited by 43
  4. 2021Bridging Global Context Interactions for High-Fidelity Image Completioncited by 40
  5. 2022ZITS++: Image Inpainting by Improving the Incremental Transformer on Structural Priorscited by 21
  6. 2023Towards Enhanced Image Inpainting: Mitigating Unwanted Object Insertion and Preserving Color Consistencycited by 19
  7. 2022AutoLink: Self-supervised Learning of Human Skeletons and Object Outlines by Linking Keypointscited by 17
  8. 2022Learning Prior Feature and Attention Enhanced Image Inpaintingcited by 16
  9. 2022Unbiased Multi-Modality Guidance for Image Inpaintingcited by 10
  10. 2024HINT: High-quality INPainting Transformer with Mask-Aware Encoding and Enhanced Attentioncited by 10
  11. 2024Semantically-aware Neural Radiance Fields for Visual Scene Understanding: A Comprehensive Reviewcited by 9
  12. 2022A Saliency-Guided Street View Image Inpainting Framework for Efficient Last-Meters Wayfindingcited by 4
13 more
  1. 2022Learning Continuous Implicit Representation for Near-Periodic Patternscited by 4
  2. 2023Towards Interactive Image Inpainting via Sketch Refinementcited by 4
  3. 2024Object Remover Performance Evaluation Methods using Class-wise Object Removal Imagescited by 3
  4. 2022A Feasibility Study on Image Inpainting for Non-cleft Lip Generation from Patients with Cleft Lipcited by 2
  5. 2022A Structure-Guided Diffusion Model for Large-Hole Image Completioncited by 1
  6. 2023Scene Structure Guidance Network: Unfolding Graph Partitioning into Pixel-Wise Feature Learningcited by 1
  7. 2021Auxiliary Loss Reweighting for Image Inpaintingcited by 0
  8. 2022Wavelet Prior Attention Learning in Axial Inpainting Networkcited by 0
  9. 2023AURA : Automatic Mask Generator using Randomized Input Sampling for Object Removalcited by 0
  10. 2024Multi-scale Semantic Prior Features Guided Deep Neural Network for Urban Street-view Imagecited by 0
  11. 2024Task-Decoupled Image Inpainting Framework for Class-specific Object Removercited by 0
  12. 2025High-Fidelity Image Inpainting with Multimodal Guided GAN Inversioncited by 0
  13. 2026Aligned Stable Inpainting: Mitigating Unwanted Object Insertion and Preserving Color Consistencycited by 0