# Semantic-Guided Inpainting Network for Complex Urban Scenes Manipulation

arXiv: https://arxiv.org/abs/2010.09334 · HTML: https://mlsanity.com/paper/2010.09334/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Oct 2020.

## Built on by (most cited first, 6 of 6)

- [Click to Move: Controlling Video Generation with Sparse Motion](https://mlsanity.com/paper/2108.08815/) (2021)
- [Unbiased Multi-Modality Guidance for Image Inpainting](https://mlsanity.com/paper/2208.11844/) (2022)
- [Shape-guided Object Inpainting](https://mlsanity.com/paper/2204.07845/) (2022)
- [Nucleus-aware Self-supervised Pretraining Using Unpaired Image-to-image Translation for Histopathology Images](https://mlsanity.com/paper/2309.07394/) (2023)
- [DeepDR: Deep Structure-Aware RGB-D Inpainting for Diminished Reality](https://mlsanity.com/paper/2312.00532/) (2023)
- [High-Fidelity Image Inpainting with Multimodal Guided GAN Inversion](https://mlsanity.com/paper/2504.12844/) (2025)

## Abstract (arXiv)

Manipulating images of complex scenes to reconstruct, insert and/or remove specific object instances is a challenging task. Complex scenes contain multiple semantics and objects, which are frequently cluttered or ambiguous, thus hampering the performance of inpainting models. Conventional techniques often rely on structural information such as object contours in multi-stage approaches that generate unreliable results and boundaries. In this work, we propose a novel deep learning model to alter a complex urban scene by removing a user-specified portion of the image and coherently inserting a new object (e.g. car or pedestrian) in that scene. Inspired by recent works on image inpainting, our proposed method leverages the semantic segmentation to model the content and structure of the image, and learn the best shape and location of the object to insert. To generate reliable results, we design a new decoder block that combines the semantic segmentation and generation task to guide better the generation of new objects and scenes, which have to be semantically consistent with the image. Our experiments, conducted on two large-scale datasets of urban scenes (Cityscapes and Indian Driving), show that our proposed approach successfully address the problem of semantically-guided inpainting of complex urban scene.