# Deep Saliency Prior for Reducing Visual Distraction

arXiv: https://arxiv.org/abs/2109.01980 · HTML: https://mlsanity.com/paper/2109.01980/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 46 references that are in the map.
- First posted on arXiv 5 Sep 2021.

## Builds on (most cited first, 17 of 17)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop](https://mlsanity.com/paper/1506.03365/) (2015)
- [Deep Image Prior](https://mlsanity.com/paper/1711.10925/) (2017)
- [Closed-Form Factorization of Latent Semantics in GANs](https://mlsanity.com/paper/2007.06600/) (2020)
- [Conditional Convolutions for Instance Segmentation](https://mlsanity.com/paper/2003.05664/) (2020)
- [What do different evaluation metrics tell us about saliency models?](https://mlsanity.com/paper/1604.03605/) (2016)
- [Deep Gaze I: Boosting Saliency Prediction with Feature Maps Trained on ImageNet](https://mlsanity.com/paper/1411.1045/) (2014)
- [EML-NET:An Expandable Multi-Layer NETwork for Saliency Prediction](https://mlsanity.com/paper/1805.01047/) (2018)
- [Improving Inversion and Generation Diversity in StyleGAN using a Gaussianized Latent Space](https://mlsanity.com/paper/2009.06529/) (2020)
- [Learning Visual Importance for Graphic Designs and Data Visualizations](https://mlsanity.com/paper/1708.02660/) (2017)
- [Tidying Deep Saliency Prediction Architectures](https://mlsanity.com/paper/2003.04942/) (2020)
- [Look here! A parametric learning based approach to redirect visual attention](https://mlsanity.com/paper/2008.05413/) (2020)
- [Paying More Attention to Saliency: Image Captioning with Saliency and Context Attention](https://mlsanity.com/paper/1706.08474/) (2017)
- [Saliency Driven Image Manipulation](https://mlsanity.com/paper/1612.02184/) (2016)
- [Guiding human gaze with convolutional neural networks](https://mlsanity.com/paper/1712.06492/) (2017)

## Built on by (most cited first, 17 of 17)

- [Interpreting CLIP's Image Representation via Text-Based Decomposition](https://mlsanity.com/paper/2310.05916/) (2023)
- [Sketch-Guided Text-to-Image Diffusion Models](https://mlsanity.com/paper/2211.13752/) (2022)
- [Realistic Saliency Guided Image Enhancement](https://mlsanity.com/paper/2306.06092/) (2023)
- [Rapid Salient Object Detection with Difference Convolutional Neural Networks](https://mlsanity.com/paper/2507.01182/) (2025)
- [Trends, Applications, and Challenges in Human Attention Modelling](https://mlsanity.com/paper/2402.18673/) (2024)
- [Salient Conditional Diffusion for Defending Against Backdoor Attacks](https://mlsanity.com/paper/2301.13862/) (2023)
- [UniAR: A Unified model for predicting human Attention and Responses on visual content](https://mlsanity.com/paper/2312.10175/) (2023)
- [SimpSON: Simplifying Photo Cleanup with Single-Click Distracting Object Segmentation Network](https://mlsanity.com/paper/2305.17624/) (2023)
- [Data Augmentation via Latent Diffusion for Saliency Prediction](https://mlsanity.com/paper/2409.07307/) (2024)
- [A Saliency-Guided Street View Image Inpainting Framework for Efficient Last-Meters Wayfinding](https://mlsanity.com/paper/2205.06934/) (2022)
- [GazeFusion: Saliency-Guided Image Generation](https://mlsanity.com/paper/2407.04191/) (2024)
- [Attention for Robot Touch: Tactile Saliency Prediction for Robust Sim-to-Real Tactile Control](https://mlsanity.com/paper/2307.14510/) (2023)
- [Varying Manifolds in Diffusion: From Time-varying Geometries to Visual Saliency](https://mlsanity.com/paper/2406.18588/) (2024)
- [Road Redesign Technique Achieving Enhanced Road Safety by Inpainting with a Diffusion Model](https://mlsanity.com/paper/2302.07440/) (2023)
- [WinDB: HMD-free and Distortion-free Panoptic Video Fixation Learning](https://mlsanity.com/paper/2305.13901/) (2023)
- [Computational Long Exposure Mobile Photography](https://mlsanity.com/paper/2308.01379/) (2023)
- [Dotting the Eye: An Intent-Driven Image Retouching Agent for Visual Focus Enhancement](https://mlsanity.com/paper/2609.01148/) (2026)

## Abstract (arXiv)

Using only a model that was trained to predict where people look at images, and no additional training data, we can produce a range of powerful editing effects for reducing distraction in images. Given an image and a mask specifying the region to edit, we backpropagate through a state-of-the-art saliency model to parameterize a differentiable editing operator, such that the saliency within the masked region is reduced. We demonstrate several operators, including: a recoloring operator, which learns to apply a color transform that camouflages and blends distractors into their surroundings; a warping operator, which warps less salient image regions to cover distractors, gradually collapsing objects into themselves and effectively removing them (an effect akin to inpainting); a GAN operator, which uses a semantic prior to fully replace image regions with plausible, less salient alternatives. The resulting effects are consistent with cognitive research on the human visual system (e.g., since color mismatch is salient, the recoloring operator learns to harmonize objects' colors with their surrounding to reduce their saliency), and, importantly, are all achieved solely through the guidance of the pretrained saliency model, with no additional supervision. We present results on a variety of natural images and conduct a perceptual study to evaluate and validate the changes in viewers' eye-gaze between the original images and our edited results.