# Blind VQA on 360deg Video via Progressively Learning from Pixels, Frames and Video

arXiv: https://arxiv.org/abs/2111.09503 · HTML: https://mlsanity.com/paper/2111.09503/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 80 references that are in the map.
- First posted on arXiv 18 Nov 2021.

## Builds on (most cited first, 27 of 27)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [CBAM: Convolutional Block Attention Module](https://mlsanity.com/paper/1807.06521/) (2018)
- [Learning Deep Features for Discriminative Localization](https://mlsanity.com/paper/1512.04150/) (2015)
- [Grad-CAM++: Improved Visual Explanations for Deep Convolutional Networks](https://mlsanity.com/paper/1710.11063/) (2017)
- [Selective Kernel Networks](https://mlsanity.com/paper/1903.06586/) (2019)
- [Blind Image Quality Assessment Using A Deep Bilinear Convolutional Neural Network](https://mlsanity.com/paper/1907.02665/) (2019)
- [Recurrent Neural Network Regularization](https://mlsanity.com/paper/1409.2329/) (2014)
- [Quality Assessment of In-the-Wild Videos](https://mlsanity.com/paper/1908.00375/) (2019)
- [MetaIQA: Deep Meta-learning for No-Reference Image Quality Assessment](https://mlsanity.com/paper/2004.05508/) (2020)
- [UGC-VQA: Benchmarking Blind Video Quality Assessment for User Generated Content](https://mlsanity.com/paper/2005.14354/) (2020)
- [Large-Scale Study of Perceptual Video Quality](https://mlsanity.com/paper/1803.01761/) (2018)
- [dipIQ: Blind Image Quality Assessment by Learning-to-Rank Discriminable Image Pairs](https://mlsanity.com/paper/1904.06505/) (2019)
- [Unified Quality Assessment of In-the-Wild Videos with Mixed Datasets Training](https://mlsanity.com/paper/2011.04263/) (2020)
- [GIQA: Generated Image Quality Assessment](https://mlsanity.com/paper/2003.08932/) (2020)
- [Blind Omnidirectional Image Quality Assessment with Viewport Oriented Graph Convolutional Networks](https://mlsanity.com/paper/2002.09140/) (2020)
- [Assessing Image Quality Issues for Real-World Problems](https://mlsanity.com/paper/2003.12511/) (2020)
- [Blind Predicting Similar Quality Map for Image Quality Assessment](https://mlsanity.com/paper/1805.08493/) (2018)
- [GTA: Global Temporal Attention for Video Action Understanding](https://mlsanity.com/paper/2012.08510/) (2020)
- [Bridge the Gap Between VQA and Human Behavior on Omnidirectional Video: A Large-Scale Dataset and a Deep Learning Model](https://mlsanity.com/paper/1807.10990/) (2018)
- [Troubleshooting Blind Image Quality Models in the Wild](https://mlsanity.com/paper/2105.06747/) (2021)
- [A Perceptually Weighted Rank Correlation Indicator for Objective Image Quality Assessment](https://mlsanity.com/paper/1705.05126/) (2017)
- [Spatial Attention-based Non-reference Perceptual Quality Prediction Network for Omnidirectional Images](https://mlsanity.com/paper/2103.06116/) (2021)
- [Assessing Visual Quality of Omnidirectional Videos](https://mlsanity.com/paper/1709.06342/) (2017)
- [Saliency Prediction on Omnidirectional Images with Generative Adversarial Imitation Learning](https://mlsanity.com/paper/1904.07080/) (2019)

## Built on by (most cited first, 4 of 4)

- [ResVR: Joint Rescaling and Viewport Rendering of Omnidirectional Images](https://mlsanity.com/paper/2404.16825/) (2024)
- [VITAL: Vision-Encoder-centered Pre-training for LMMs in Visual Quality Assessment](https://mlsanity.com/paper/2511.17962/) (2025)
- [Research on Audio-Visual Quality Assessment Dataset and Method for User-Generated Omnidirectional Video](https://mlsanity.com/paper/2506.10331/) (2025)
- [LLaVA-Assessor: Building the Foundation LMM For Visual Quality Assessment](https://mlsanity.com/paper/2609.26205/) (2026)

## Abstract (arXiv)

Blind visual quality assessment (BVQA) on 360textdegree video plays a key role in optimizing immersive multimedia systems. When assessing the quality of 360textdegree video, human tends to perceive its quality degradation from the viewport-based spatial distortion of each spherical frame to motion artifact across adjacent frames, ending with the video-level quality score, i.e., a progressive quality assessment paradigm. However, the existing BVQA approaches for 360textdegree video neglect this paradigm. In this paper, we take into account the progressive paradigm of human perception towards spherical video quality, and thus propose a novel BVQA approach (namely ProVQA) for 360textdegree video via progressively learning from pixels, frames and video. Corresponding to the progressive learning of pixels, frames and video, three sub-nets are designed in our ProVQA approach, i.e., the spherical perception aware quality prediction (SPAQ), motion perception aware quality prediction (MPAQ) and multi-frame temporal non-local (MFTN) sub-nets. The SPAQ sub-net first models the spatial quality degradation based on spherical perception mechanism of human. Then, by exploiting motion cues across adjacent frames, the MPAQ sub-net properly incorporates motion contextual information for quality assessment on 360textdegree video. Finally, the MFTN sub-net aggregates multi-frame quality degradation to yield the final quality score, via exploring long-term quality correlation from multiple frames. The experiments validate that our approach significantly advances the state-of-the-art BVQA performance on 360textdegree video over two datasets, the code of which has been public in urlhttps://github.com/yanglixiaoshen/ProVQA.