# ESIQA: Perceptual Quality Assessment of Vision-Pro-based Egocentric Spatial Images

arXiv: https://arxiv.org/abs/2407.21363 · HTML: https://mlsanity.com/paper/2407.21363/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 55 references that are in the map.
- First posted on arXiv 31 Jul 2024.

## Builds on (most cited first, 21 of 21)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://mlsanity.com/paper/2312.00752/) (2023)
- [Restormer: Efficient Transformer for High-Resolution Image Restoration](https://mlsanity.com/paper/2111.09881/) (2021)
- [Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality](https://mlsanity.com/paper/2405.21060/) (2024)
- [CrossViT: Cross-Attention Multi-Scale Vision Transformer for Image Classification](https://mlsanity.com/paper/2103.14899/) (2021)
- [MANIQA: Multi-dimension Attention Network for No-Reference Image Quality Assessment](https://mlsanity.com/paper/2204.08958/) (2022)
- [CMT: Convolutional Neural Networks Meet Vision Transformers](https://mlsanity.com/paper/2107.06263/) (2021)
- [Deep Neural Networks for No-Reference and Full-Reference Image Quality Assessment](https://mlsanity.com/paper/1612.01697/) (2016)
- [No-Reference Image Quality Assessment via Transformers, Relative Ranking, and Self-Consistency](https://mlsanity.com/paper/2108.06858/) (2021)
- [CAT: Cross Attention in Vision Transformer](https://mlsanity.com/paper/2106.05786/) (2021)
- [Perceptual Video Quality Assessment: A Survey](https://mlsanity.com/paper/2402.03413/) (2024)
- [Perceptual Quality Assessment of Omnidirectional Images](https://mlsanity.com/paper/2207.02674/) (2022)
- [Confusing Image Quality Assessment: Towards Better Augmented Reality Experience](https://mlsanity.com/paper/2204.04900/) (2022)
- [Saliency in Augmented Reality](https://mlsanity.com/paper/2204.08308/) (2022)
- [KonIQ-10k: Towards an ecologically valid and large-scale IQA database](https://mlsanity.com/paper/1803.08489/) (2018)
- [Perceptual Quality Assessment of Omnidirectional Audio-visual Signals](https://mlsanity.com/paper/2307.10813/) (2023)
- [VR IQA NET: Deep Virtual Reality Image Quality Assessment using Adversarial Learning](https://mlsanity.com/paper/1804.03943/) (2018)

## Built on by (most cited first, 7 of 7)

- [HarmonyIQA: Pioneering Benchmark and Model for Image Harmonization Quality Assessment](https://mlsanity.com/paper/2501.01116/) (2025)
- [What Makes for a Good Stereoscopic Image?](https://mlsanity.com/paper/2412.21127/) (2024)
- [ESVQA: Perceptual Quality Assessment of Egocentric Spatial Videos](https://mlsanity.com/paper/2412.20423/) (2024)
- [Quality Assessment and Distortion-aware Saliency Prediction for AI-Generated Omnidirectional Images](https://mlsanity.com/paper/2506.21925/) (2025)
- [Evaluating Graphical Perception Capabilities of Vision Transformers](https://mlsanity.com/paper/2602.18178/) (2026)
- [Immersive Multimedia Communication: State-of-the-Art on eXtended Reality Streaming](https://mlsanity.com/paper/2506.10004/) (2025)
- [Enhancing Image Quality Assessment Ability of LMMs via Retrieval-Augmented Generation](https://mlsanity.com/paper/2601.08311/) (2026)

## Abstract (arXiv)

With the development of eXtended Reality (XR), photo capturing and display technology based on head-mounted displays (HMDs) have experienced significant advancements and gained considerable attention. Egocentric spatial images and videos are emerging as a compelling form of stereoscopic XR content. The assessment for the Quality of Experience (QoE) of XR content is important to ensure a high-quality viewing experience. Different from traditional 2D images, egocentric spatial images present challenges for perceptual quality assessment due to their special shooting, processing methods, and stereoscopic characteristics. However, the corresponding image quality assessment (IQA) research for egocentric spatial images is still lacking. In this paper, we establish the Egocentric Spatial Images Quality Assessment Database (ESIQAD), the first IQA database dedicated for egocentric spatial images as far as we know. Our ESIQAD includes 500 egocentric spatial images and the corresponding mean opinion scores (MOSs) under three display modes, including 2D display, 3D-window display, and 3D-immersive display. Based on our ESIQAD, we propose a novel mamba2-based multi-stage feature fusion model, termed ESIQAnet, which predicts the perceptual quality of egocentric spatial images under the three display modes. Specifically, we first extract features from multiple visual state space duality (VSSD) blocks, then apply cross attention to fuse binocular view information and use transposed attention to further refine the features. The multi-stage features are finally concatenated and fed into a quality regression network to predict the quality score. Extensive experimental results demonstrate that the ESIQAnet outperforms 22 state-of-the-art IQA models on the ESIQAD under all three display modes. The database and code are available at https://github.com/IntMeGroup/ESIQA.