# TranSalNet: Towards perceptually relevant visual saliency prediction

arXiv: https://arxiv.org/abs/2110.03593 · HTML: https://mlsanity.com/paper/2110.03593/

## Key facts

- Cited by 44 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 77 references that are in the map.
- First posted on arXiv 7 Oct 2021.

## Builds on (most cited first, 30 of 31)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs](https://mlsanity.com/paper/1606.00915/) (2016)
- [Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers](https://mlsanity.com/paper/2012.15840/) (2020)
- [Learning Transferable Architectures for Scalable Image Recognition](https://mlsanity.com/paper/1707.07012/) (2017)
- [Are Sixteen Heads Really Better than One?](https://mlsanity.com/paper/1905.10650/) (2019)
- [DeepViT: Towards Deeper Vision Transformer](https://mlsanity.com/paper/2103.11886/) (2021)
- [HigherHRNet: Scale-Aware Representation Learning for Bottom-Up Human Pose Estimation](https://mlsanity.com/paper/1908.10357/) (2019)
- [What do different evaluation metrics tell us about saliency models?](https://mlsanity.com/paper/1604.03605/) (2016)
- [Visual Saliency Transformer](https://mlsanity.com/paper/2104.12099/) (2021)
- [Predicting Human Eye Fixations via an LSTM-based Saliency Attentive Model](https://mlsanity.com/paper/1611.09571/) (2016)
- [Unified Image and Video Saliency Modeling](https://mlsanity.com/paper/2003.05477/) (2020)
- [Deep Visual Attention Prediction](https://mlsanity.com/paper/1705.02544/) (2017)
- [SalGAN: Visual Saliency Prediction with Generative Adversarial Networks](https://mlsanity.com/paper/1701.01081/) (2017)
- [DeepGaze IIE: Calibrated prediction in and out-of-domain for state-of-the-art saliency modeling](https://mlsanity.com/paper/2105.12441/) (2021)
- [A Deep Multi-Level Network for Saliency Prediction](https://mlsanity.com/paper/1609.01064/) (2016)
- [CAT2000: A Large Scale Fixation Dataset for Boosting Saliency Research](https://mlsanity.com/paper/1505.03581/) (2015)
- [EML-NET:An Expandable Multi-Layer NETwork for Saliency Prediction](https://mlsanity.com/paper/1805.01047/) (2018)
- [Contextual Encoder-Decoder Network for Visual Saliency Prediction](https://mlsanity.com/paper/1902.06634/) (2019)
- [How is Gaze Influenced by Image Transformations? Dataset and Model](https://mlsanity.com/paper/1905.06803/) (2019)
- [A Deep Spatial Contextual Long-term Recurrent Convolutional Network for Saliency Detection](https://mlsanity.com/paper/1610.01708/) (2016)

## Built on by (most cited first, 30 of 44)

- [Rectify ViT Shortcut Learning by Visual Saliency](https://mlsanity.com/paper/2206.08567/) (2022)
- [SUM: Saliency Unification through Mamba for Visual Attention Modeling](https://mlsanity.com/paper/2406.17815/) (2024)
- [NTIRE 2026 Challenge on Video Saliency Prediction: Methods and Results](https://mlsanity.com/paper/2604.14816/) (2026)
- [Trends, Applications, and Challenges in Human Attention Modelling](https://mlsanity.com/paper/2402.18673/) (2024)
- [G-VOILA: Gaze-Facilitated Information Querying in Daily Scenarios](https://mlsanity.com/paper/2405.07652/) (2024)
- [How is Visual Attention Influenced by Text Guidance? Database and Model](https://mlsanity.com/paper/2404.07537/) (2024)
- [LMME3DHF: Benchmarking and Evaluating Multimodal 3D Human Face Generation with LMMs](https://mlsanity.com/paper/2504.20466/) (2025)
- [Agentic Retoucher for Text-To-Image Generation](https://mlsanity.com/paper/2601.02046/) (2026)
- [Brand Visibility in Packaging: A Deep Learning Approach for Logo Detection, Saliency-Map Prediction, and Logo Placement Analysis](https://mlsanity.com/paper/2403.02336/) (2024)
- [OAT: Object-Level Attention Transformer for Gaze Scanpath Prediction](https://mlsanity.com/paper/2407.13335/) (2024)
- [Data Augmentation via Latent Diffusion for Saliency Prediction](https://mlsanity.com/paper/2409.07307/) (2024)
- [DiffEye: Diffusion-Based Continuous Eye-Tracking Data Generation Conditioned on Natural Images](https://mlsanity.com/paper/2509.16767/) (2025)
- [A Saliency-Guided Street View Image Inpainting Framework for Efficient Last-Meters Wayfinding](https://mlsanity.com/paper/2205.06934/) (2022)
- [What Do Deep Saliency Models Learn about Visual Attention?](https://mlsanity.com/paper/2310.09679/) (2023)
- [Rethinking Object Saliency Ranking: A Novel Whole-flow Processing Paradigm](https://mlsanity.com/paper/2312.03226/) (2023)
- [RClicks: Realistic Click Simulation for Benchmarking Interactive Segmentation](https://mlsanity.com/paper/2410.11722/) (2024)
- [Modeling Saliency Dataset Bias](https://mlsanity.com/paper/2505.10169/) (2025)
- [Gaze-Informed Vision Transformers: Predicting Driving Decisions Under Uncertainty](https://mlsanity.com/paper/2308.13969/) (2023)
- [SAMIC: Segment Anything with In-Context Spatial Prompt Engineering](https://mlsanity.com/paper/2412.11998/) (2024)
- [Learning Saliency From Fixations](https://mlsanity.com/paper/2311.14073/) (2023)
- [RL-RC-DoT: A Block-level RL agent for Task-Aware Video Compression](https://mlsanity.com/paper/2501.12216/) (2025)
- [Quality Assessment and Distortion-aware Saliency Prediction for AI-Generated Omnidirectional Images](https://mlsanity.com/paper/2506.21925/) (2025)
- [Phase-Aligned RoPE for Mixed-Resolution Diffusion Transformer](https://mlsanity.com/paper/2511.19778/) (2025)
- [Image Cropping under Design Constraints](https://mlsanity.com/paper/2310.08892/) (2023)
- [Bridging the Gap Between Saliency Prediction and Image Quality Assessment](https://mlsanity.com/paper/2405.04997/) (2024)
- [MDS-ViTNet: Improving saliency prediction for Eye-Tracking with Vision Transformer](https://mlsanity.com/paper/2405.19501/) (2024)
- [SalNAS: Efficient Saliency-prediction Neural Architecture Search with self-knowledge distillation](https://mlsanity.com/paper/2407.20062/) (2024)
- [Relevance-guided Audio Visual Fusion for Video Saliency Prediction](https://mlsanity.com/paper/2411.11454/) (2024)
- [Tell Me Without Telling Me: Two-Way Prediction of Visualization Literacy and Visual Attention](https://mlsanity.com/paper/2508.03713/) (2025)
- [SPGen: Stochastic scanpath generation for paintings using unsupervised domain adaptation](https://mlsanity.com/paper/2602.22049/) (2026)

## Abstract (arXiv)

Visual saliency prediction using transformers - Convolutional neural networks (CNNs) have significantly advanced computational modelling for saliency prediction. However, accurately simulating the mechanisms of visual attention in the human cortex remains an academic challenge. It is critical to integrate properties of human vision into the design of CNN architectures, leading to perceptually more relevant saliency prediction. Due to the inherent inductive biases of CNN architectures, there is a lack of sufficient long-range contextual encoding capacity. This hinders CNN-based saliency models from capturing properties that emulate viewing behaviour of humans. Transformers have shown great potential in encoding long-range information by leveraging the self-attention mechanism. In this paper, we propose a novel saliency model that integrates transformer components to CNNs to capture the long-range contextual visual information. Experimental results show that the transformers provide added value to saliency prediction, enhancing its perceptual relevance in the performance. Our proposed saliency model using transformers has achieved superior results on public benchmarks and competitions for saliency prediction models. The source code of our proposed saliency model TranSalNet is available at: https://github.com/LJOVO/TranSalNet