# A Saliency-Guided Street View Image Inpainting Framework for Efficient Last-Meters Wayfinding

arXiv: https://arxiv.org/abs/2205.06934 · HTML: https://mlsanity.com/paper/2205.06934/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 72 references that are in the map.
- First posted on arXiv 14 May 2022.

## Builds on (most cited first, 29 of 29)

- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1802.02611/) (2018)
- [Pyramid Scene Parsing Network](https://mlsanity.com/paper/1612.01105/) (2016)
- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs](https://mlsanity.com/paper/1711.11585/) (2017)
- [Resolution-robust Large Mask Inpainting with Fourier Convolutions](https://mlsanity.com/paper/2109.07161/) (2021)
- [Free-Form Image Inpainting with Gated Convolution](https://mlsanity.com/paper/1806.03589/) (2018)
- [Generative Image Inpainting with Contextual Attention](https://mlsanity.com/paper/1801.07892/) (2018)
- [Image Inpainting for Irregular Holes Using Partial Convolutions](https://mlsanity.com/paper/1804.07723/) (2018)
- [U2-Net: Going Deeper with Nested U-Structure for Salient Object Detection](https://mlsanity.com/paper/2005.09007/) (2020)
- [Pluralistic Image Completion](https://mlsanity.com/paper/1903.04227/) (2019)
- [Contextual Residual Aggregation for Ultra High-Resolution Image Inpainting](https://mlsanity.com/paper/2005.09704/) (2020)
- [Recurrent Feature Reasoning for Image Inpainting](https://mlsanity.com/paper/2008.03737/) (2020)
- [Unified Image and Video Saliency Modeling](https://mlsanity.com/paper/2003.05477/) (2020)
- [Revisiting Video Saliency: A Large-scale Benchmark and a New Model](https://mlsanity.com/paper/1801.07424/) (2018)
- [Review of Visual Saliency Detection with Comprehensive Information](https://mlsanity.com/paper/1803.03391/) (2018)
- [TranSalNet: Towards perceptually relevant visual saliency prediction](https://mlsanity.com/paper/2110.03593/) (2021)
- [Recent progress in semantic image segmentation](https://mlsanity.com/paper/1809.10198/) (2018)
- [Learning a Sketch Tensor Space for Image Inpainting of Man-made Scenes](https://mlsanity.com/paper/2103.15087/) (2021)
- [Privacy Protection in Street-View Panoramas using Depth and Multi-View Imagery](https://mlsanity.com/paper/1903.11532/) (2019)
- [Deep Saliency Prior for Reducing Visual Distraction](https://mlsanity.com/paper/2109.01980/) (2021)
- [Understanding and Visualizing Deep Visual Saliency Models](https://mlsanity.com/paper/1903.02501/) (2019)
- [Look here! A parametric learning based approach to redirect visual attention](https://mlsanity.com/paper/2008.05413/) (2020)
- [Saliency Driven Image Manipulation](https://mlsanity.com/paper/1612.02184/) (2016)
- [Empty Cities: Image Inpainting for a Dynamic-Object-Invariant Space](https://mlsanity.com/paper/1809.10239/) (2018)
- [Multi Sensor Fusion for Navigation and Mapping in Autonomous Vehicles: Accurate Localization in Urban Environments](https://mlsanity.com/paper/2103.13719/) (2021)
- [LiDAR-assisted Large-scale Privacy Protection in Street-view Cycloramas](https://mlsanity.com/paper/1903.05598/) (2019)

## Built on by (most cited first, 4 of 4)

- [Coverage and Bias of Street View Imagery in Mapping the Urban Environment](https://mlsanity.com/paper/2409.15386/) (2024)
- [UPDExplainer: an Interpretable Transformer-based Framework for Urban Physical Disorder Detection Using Street View Imagery](https://mlsanity.com/paper/2305.02911/) (2023)
- [Dense Feature Interaction Network for Image Inpainting Localization](https://mlsanity.com/paper/2408.02191/) (2024)
- [URSimulator: Human-Perception-Driven Prompt Tuning for Enhanced Virtual Urban Renewal via Diffusion Models](https://mlsanity.com/paper/2409.14589/) (2024)

## Abstract (arXiv)

Global Positioning Systems (GPS) have played a crucial role in various navigation applications. Nevertheless, localizing the perfect destination within the last few meters remains an important but unresolved problem. Limited by the GPS positioning accuracy, navigation systems always show users a vicinity of a destination, but not its exact location. Street view images (SVI) in maps as an immersive media technology have served as an aid to provide the physical environment for human last-meters wayfinding. However, due to the large diversity of geographic context and acquisition conditions, the captured SVI always contains various distracting objects (e.g., pedestrians and vehicles), which will distract human visual attention from efficiently finding the destination in the last few meters. To address this problem, we highlight the importance of reducing visual distraction in image-based wayfinding by proposing a saliency-guided image inpainting framework. It aims at redirecting human visual attention from distracting objects to destination-related objects for more efficient and accurate wayfinding in the last meters. Specifically, a context-aware distracting object detection method driven by deep salient object detection has been designed to extract distracting objects from three semantic levels in SVI. Then we employ a large-mask inpainting method with fast Fourier convolutions to remove the detected distracting objects. Experimental results with both qualitative and quantitative analysis show that our saliency-guided inpainting method can not only achieve great perceptual quality in street view images but also redirect the human's visual attention to focus more on static location-related objects than distracting ones. The human-based evaluation also justified the effectiveness of our method in improving the efficiency of locating the target destination.