# Saliency in Augmented Reality

arXiv: https://arxiv.org/abs/2204.08308 · HTML: https://mlsanity.com/paper/2204.08308/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 66 references that are in the map.
- First posted on arXiv 18 Apr 2022.

## Builds on (most cited first, 16 of 16)

- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [Taming Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2012.09841/) (2020)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [Ego4D: Around the World in 3,000 Hours of Egocentric Video](https://mlsanity.com/paper/2110.07058/) (2021)
- [What do different evaluation metrics tell us about saliency models?](https://mlsanity.com/paper/1604.03605/) (2016)
- [Predicting Human Eye Fixations via an LSTM-based Saliency Attentive Model](https://mlsanity.com/paper/1611.09571/) (2016)
- [SalGAN: Visual Saliency Prediction with Generative Adversarial Networks](https://mlsanity.com/paper/1701.01081/) (2017)
- [A Deep Multi-Level Network for Saliency Prediction](https://mlsanity.com/paper/1609.01064/) (2016)
- [DeepFix: A Fully Convolutional Neural Network for predicting Human Eye Fixations](https://mlsanity.com/paper/1510.02927/) (2015)
- [End-to-End Human-Gaze-Target Detection with Transformers](https://mlsanity.com/paper/2203.10433/) (2022)
- [Confusing Image Quality Assessment: Towards Better Augmented Reality Experience](https://mlsanity.com/paper/2204.04900/) (2022)
- [How is Gaze Influenced by Image Transformations? Dataset and Model](https://mlsanity.com/paper/1905.06803/) (2019)
- [Visual Saliency Based on Scale-Space Analysis in the Frequency Domain](https://mlsanity.com/paper/1605.01999/) (2016)
- [Saliency Prediction on Omnidirectional Images with Generative Adversarial Imitation Learning](https://mlsanity.com/paper/1904.07080/) (2019)
- [Toward Better Understanding of Saliency Prediction in Augmented 360 Degree Videos](https://mlsanity.com/paper/1912.05971/) (2019)

## Built on by (most cited first, 17 of 17)

- [AIGCIQA2023: A Large-scale Image Quality Assessment Database for AI Generated Images: from the Perspectives of Quality, Authenticity and Correspondence](https://mlsanity.com/paper/2307.00211/) (2023)
- [Perceptual Video Quality Assessment: A Survey](https://mlsanity.com/paper/2402.03413/) (2024)
- [Generative AI-empowered Simulation for Autonomous Driving in Vehicular Mixed Reality Metaverses](https://mlsanity.com/paper/2302.08418/) (2023)
- [AIGCOIQA2024: Perceptual Quality Assessment of AI Generated Omnidirectional Images](https://mlsanity.com/paper/2404.01024/) (2024)
- [ViDDAR: Vision Language Model-Based Task-Detrimental Content Detection for Augmented Reality](https://mlsanity.com/paper/2501.12553/) (2025)
- [Quality Assessment for AI Generated Images with Instruction Tuning](https://mlsanity.com/paper/2405.07346/) (2024)
- [Perceptual Quality Assessment of Omnidirectional Audio-visual Signals](https://mlsanity.com/paper/2307.10813/) (2023)
- [Audio-visual Saliency for Omnidirectional Videos](https://mlsanity.com/paper/2311.05190/) (2023)
- [How is Visual Attention Influenced by Text Guidance? Database and Model](https://mlsanity.com/paper/2404.07537/) (2024)
- [ESIQA: Perceptual Quality Assessment of Vision-Pro-based Egocentric Spatial Images](https://mlsanity.com/paper/2407.21363/) (2024)
- [Joint Gaze-Location and Gaze-Object Detection](https://mlsanity.com/paper/2308.13857/) (2023)
- [How Does Audio Influence Visual Attention in Omnidirectional Videos? Database and Model](https://mlsanity.com/paper/2408.05411/) (2024)
- [MVBIND: Self-Supervised Music Recommendation For Videos Via Embedding Space Binding](https://mlsanity.com/paper/2405.09286/) (2024)
- [Quality-guided Skin Tone Enhancement for Portrait Photography](https://mlsanity.com/paper/2406.15848/) (2024)
- [Multi-Dimensional Quality Assessment for Text-to-3D Assets: Dataset and Model](https://mlsanity.com/paper/2502.16915/) (2025)
- [Quality Assessment and Distortion-aware Saliency Prediction for AI-Generated Omnidirectional Images](https://mlsanity.com/paper/2506.21925/) (2025)
- [SalFAU-Net: Saliency Fusion Attention U-Net for Salient Object Detection](https://mlsanity.com/paper/2405.02906/) (2024)

## Abstract (arXiv)

With the rapid development of multimedia technology, Augmented Reality (AR) has become a promising next-generation mobile platform. The primary theory underlying AR is human visual confusion, which allows users to perceive the real-world scenes and augmented contents (virtual-world scenes) simultaneously by superimposing them together. To achieve good Quality of Experience (QoE), it is important to understand the interaction between two scenarios, and harmoniously display AR contents. However, studies on how this superimposition will influence the human visual attention are lacking. Therefore, in this paper, we mainly analyze the interaction effect between background (BG) scenes and AR contents, and study the saliency prediction problem in AR. Specifically, we first construct a Saliency in AR Dataset (SARD), which contains 450 BG images, 450 AR images, as well as 1350 superimposed images generated by superimposing BG and AR images in pair with three mixing levels. A large-scale eye-tracking experiment among 60 subjects is conducted to collect eye movement data. To better predict the saliency in AR, we propose a vector quantized saliency prediction method and generalize it for AR saliency prediction. For comparison, three benchmark methods are proposed and evaluated together with our proposed method on our SARD. Experimental results demonstrate the superiority of our proposed method on both of the common saliency prediction problem and the AR saliency prediction problem over benchmark methods. Our dataset and code are available at: https://github.com/DuanHuiyu/ARSaliency.