# No-Reference Image Quality Assessment via Transformers, Relative Ranking, and Self-Consistency

arXiv: https://arxiv.org/abs/2108.06858 · HTML: https://mlsanity.com/paper/2108.06858/

## Key facts

- Cited by 166 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 84 references that are in the map.
- First posted on arXiv 16 Aug 2021.

## Builds on (most cited first, 30 of 37)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [Visualizing and Understanding Convolutional Networks](https://mlsanity.com/paper/1311.2901/) (2013)
- [Group Equivariant Convolutional Networks](https://mlsanity.com/paper/1602.07576/) (2016)
- [Pre-Trained Image Processing Transformer](https://mlsanity.com/paper/2012.00364/) (2020)
- [Image Quality Assessment: Unifying Structure and Texture Similarity](https://mlsanity.com/paper/2004.07728/) (2020)
- [Image Transformer](https://mlsanity.com/paper/1802.05751/) (2018)
- [NIMA: Neural Image Assessment](https://mlsanity.com/paper/1709.05424/) (2017)
- [Attention Augmented Convolutional Networks](https://mlsanity.com/paper/1904.09925/) (2019)
- [KonIQ-10k: An ecologically valid database for deep learning of blind image quality assessment](https://mlsanity.com/paper/1910.06180/) (2019)
- [Blind Image Quality Assessment Using A Deep Bilinear Convolutional Neural Network](https://mlsanity.com/paper/1907.02665/) (2019)
- [Harmonic Networks: Deep Translation and Rotation Equivariance](https://mlsanity.com/paper/1612.04642/) (2016)
- [Deep Neural Networks for No-Reference and Full-Reference Image Quality Assessment](https://mlsanity.com/paper/1612.01697/) (2016)
- [From Patches to Pictures (PaQ-2-PiQ): Mapping the Perceptual Space of Picture Quality](https://mlsanity.com/paper/1912.10088/) (2019)
- [Massive Online Crowdsourced Study of Subjective and Objective Picture Quality](https://mlsanity.com/paper/1511.02919/) (2015)
- [RankIQA: Learning from Rankings for No-reference Image Quality Assessment](https://mlsanity.com/paper/1707.08347/) (2017)
- [MetaIQA: Deep Meta-learning for No-Reference Image Quality Assessment](https://mlsanity.com/paper/2004.05508/) (2020)
- [Uncertainty-Aware Blind Image Quality Assessment in the Laboratory and Wild](https://mlsanity.com/paper/2005.13983/) (2020)
- [Robust Learning Through Cross-Task Consistency](https://mlsanity.com/paper/2006.04096/) (2020)
- [Transformer for Image Quality Assessment](https://mlsanity.com/paper/2101.01097/) (2020)
- [dipIQ: Blind Image Quality Assessment by Learning-to-Rank Discriminable Image Pairs](https://mlsanity.com/paper/1904.06505/) (2019)
- [Perceptual Quality Prediction on Authentically Distorted Images Using a Bag of Features Approach](https://mlsanity.com/paper/1609.04757/) (2016)
- [Hallucinated-IQA: No-Reference Image Quality Assessment via Adversarial Learning](https://mlsanity.com/paper/1804.01681/) (2018)

## Built on by (most cited first, 30 of 166)

- [MANIQA: Multi-dimension Attention Network for No-Reference Image Quality Assessment](https://mlsanity.com/paper/2204.08958/) (2022)
- [Blind Image Quality Assessment via Vision-Language Correspondence: A Multitask Learning Perspective](https://mlsanity.com/paper/2303.14968/) (2023)
- [TOPIQ: A Top-down Approach from Semantics to Distortions for Image Quality Assessment](https://mlsanity.com/paper/2308.03060/) (2023)
- [AGIQA-3K: An Open Database for AI-Generated Image Quality Assessment](https://mlsanity.com/paper/2306.04717/) (2023)
- [ARNIQA: Learning Distortion Manifold for Image Quality Assessment](https://mlsanity.com/paper/2310.14918/) (2023)
- [Towards Open-ended Visual Quality Comparison](https://mlsanity.com/paper/2402.16641/) (2024)
- [Content-based Unrestricted Adversarial Attack](https://mlsanity.com/paper/2305.10665/) (2023)
- [Adaptive Image Quality Assessment via Teaching Large Multimodal Model to Compare](https://mlsanity.com/paper/2405.19298/) (2024)
- [Data-Efficient Image Quality Assessment with Attention-Panel Decoder](https://mlsanity.com/paper/2304.04952/) (2023)
- [NTIRE 2022 Challenge on Perceptual Image Quality Assessment](https://mlsanity.com/paper/2206.11695/) (2022)
- [Light the Night: A Multi-Condition Diffusion Framework for Unpaired Low-Light Enhancement in Autonomous Driving](https://mlsanity.com/paper/2404.04804/) (2024)
- [GoodDrag: Towards Good Practices for Drag Editing with Diffusion Models](https://mlsanity.com/paper/2404.07206/) (2024)
- [Quality-Aware Image-Text Alignment for Opinion-Unaware Image Quality Assessment](https://mlsanity.com/paper/2403.11176/) (2024)
- [Large Multi-modality Model Assisted AI-Generated Image Quality Assessment](https://mlsanity.com/paper/2404.17762/) (2024)
- [HoloDreamer: Holistic 3D Panoramic World Generation from Text Descriptions](https://mlsanity.com/paper/2407.15187/) (2024)
- [LMM4LMM: Benchmarking and Evaluating Large-multimodal Image Generation with LMMs](https://mlsanity.com/paper/2504.08358/) (2025)
- [NightHaze: Nighttime Image Dehazing via Self-Prior Learning](https://mlsanity.com/paper/2403.07408/) (2024)
- [Going the Extra Mile in Face Image Quality Assessment: A Novel Database and Model](https://mlsanity.com/paper/2207.04904/) (2022)
- [NTIRE 2024 Quality Assessment of AI-Generated Content Challenge](https://mlsanity.com/paper/2404.16687/) (2024)
- [PTM-VQA: Efficient Video Quality Assessment Leveraging Diverse PreTrained Models from the Wild](https://mlsanity.com/paper/2405.17765/) (2024)
- [DSL-FIQA: Assessing Facial Image Quality via Dual-Set Degradation Learning and Landmark-Guided Transformer](https://mlsanity.com/paper/2406.09622/) (2024)
- [AIGCOIQA2024: Perceptual Quality Assessment of AI Generated Omnidirectional Images](https://mlsanity.com/paper/2404.01024/) (2024)
- [Quality-aware Pre-trained Models for Blind Image Quality Assessment](https://mlsanity.com/paper/2303.00521/) (2023)
- [Video Quality Assessment: A Comprehensive Survey](https://mlsanity.com/paper/2412.04508/) (2024)
- [An Image Quality Assessment Dataset for Portraits](https://mlsanity.com/paper/2304.05772/) (2023)
- [Comparing the Robustness of Modern No-Reference Image- and Video-Quality Metrics to Adversarial Attacks](https://mlsanity.com/paper/2310.06958/) (2023)
- [DP-IQA: Utilizing Diffusion Prior for Blind Image Quality Assessment in the Wild](https://mlsanity.com/paper/2405.19996/) (2024)
- [Quality Assessment for AI Generated Images with Instruction Tuning](https://mlsanity.com/paper/2405.07346/) (2024)
- [UniQA: Unified Vision-Language Pre-training for Image Quality and Aesthetic Assessment](https://mlsanity.com/paper/2406.01069/) (2024)
- [Benchmarking Multi-dimensional AIGC Video Quality Assessment: A Dataset and Unified Model](https://mlsanity.com/paper/2407.21408/) (2024)

## Abstract (arXiv)

The goal of No-Reference Image Quality Assessment (NR-IQA) is to estimate the perceptual image quality in accordance with subjective evaluations, it is a complex and unsolved problem due to the absence of the pristine reference image. In this paper, we propose a novel model to address the NR-IQA task by leveraging a hybrid approach that benefits from Convolutional Neural Networks (CNNs) and self-attention mechanism in Transformers to extract both local and non-local features from the input image. We capture local structure information of the image via CNNs, then to circumvent the locality bias among the extracted CNNs features and obtain a non-local representation of the image, we utilize Transformers on the extracted features where we model them as a sequential input to the Transformer model. Furthermore, to improve the monotonicity correlation between the subjective and objective scores, we utilize the relative distance information among the images within each batch and enforce the relative ranking among them. Last but not least, we observe that the performance of NR-IQA models degrades when we apply equivariant transformations (e.g. horizontal flipping) to the inputs. Therefore, we propose a method that leverages self-consistency as a source of self-supervision to improve the robustness of NRIQA models. Specifically, we enforce self-consistency between the outputs of our quality assessment model for each image and its transformation (horizontally flipped) to utilize the rich self-supervisory information and reduce the uncertainty of the model. To demonstrate the effectiveness of our work, we evaluate it on seven standard IQA datasets (both synthetic and authentic) and show that our model achieves state-of-the-art results on various datasets.