# Saliency Prediction with External Knowledge

arXiv: https://arxiv.org/abs/2007.13839 · HTML: https://mlsanity.com/paper/2007.13839/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Jul 2020.

## Built on by (most cited first, 5 of 5)

- [TempSAL -- Uncovering Temporal Information for Deep Saliency Prediction](https://mlsanity.com/paper/2301.02315/) (2023)
- [Graph-based Knowledge Distillation: A survey and experimental evaluation](https://mlsanity.com/paper/2302.14643/) (2023)
- [Data Augmentation via Latent Diffusion for Saliency Prediction](https://mlsanity.com/paper/2409.07307/) (2024)
- [Modeling Object Dissimilarity for Deep Saliency Prediction](https://mlsanity.com/paper/2104.03864/) (2021)
- [Audio-Visual Collaborative Representation Learning for Dynamic Saliency Prediction](https://mlsanity.com/paper/2109.08371/) (2021)

## Abstract (arXiv)

The last decades have seen great progress in saliency prediction, with the success of deep neural networks that are able to encode high-level semantics. Yet, while humans have the innate capability in leveraging their knowledge to decide where to look (e.g. people pay more attention to familiar faces such as celebrities), saliency prediction models have only been trained with large eye-tracking datasets. This work proposes to bridge this gap by explicitly incorporating external knowledge for saliency models as humans do. We develop networks that learn to highlight regions by incorporating prior knowledge of semantic relationships, be it general or domain-specific, depending on the task of interest. At the core of the method is a new Graph Semantic Saliency Network (GraSSNet) that constructs a graph that encodes semantic relationships learned from external knowledge. A Spatial Graph Attention Network is then developed to update saliency features based on the learned graph. Experiments show that the proposed model learns to predict saliency from the external knowledge and outperforms the state-of-the-art on four saliency benchmarks.