# Learning Transformer Features for Image Quality Assessment

arXiv: https://arxiv.org/abs/2112.00485 · HTML: https://mlsanity.com/paper/2112.00485/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 45 references that are in the map.
- First posted on arXiv 1 Dec 2021.

## Builds on (most cited first, 26 of 26)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [MUSIQ: Multi-scale Image Quality Transformer](https://mlsanity.com/paper/2108.05997/) (2021)
- [Image Quality Assessment: Unifying Structure and Texture Similarity](https://mlsanity.com/paper/2004.07728/) (2020)
- [NIMA: Neural Image Assessment](https://mlsanity.com/paper/1709.05424/) (2017)
- [KonIQ-10k: An ecologically valid database for deep learning of blind image quality assessment](https://mlsanity.com/paper/1910.06180/) (2019)
- [Deep Neural Networks for No-Reference and Full-Reference Image Quality Assessment](https://mlsanity.com/paper/1612.01697/) (2016)
- [Gradient Magnitude Similarity Deviation: A Highly Efficient Perceptual Image Quality Index](https://mlsanity.com/paper/1308.3052/) (2013)
- [No-Reference Image Quality Assessment via Transformers, Relative Ranking, and Self-Consistency](https://mlsanity.com/paper/2108.06858/) (2021)
- [PieAPP: Perceptual Image-Error Assessment through Pairwise Preference](https://mlsanity.com/paper/1806.02067/) (2018)
- [RankIQA: Learning from Rankings for No-reference Image Quality Assessment](https://mlsanity.com/paper/1707.08347/) (2017)
- [MetaIQA: Deep Meta-learning for No-Reference Image Quality Assessment](https://mlsanity.com/paper/2004.05508/) (2020)
- [Transformer for Image Quality Assessment](https://mlsanity.com/paper/2101.01097/) (2020)
- [dipIQ: Blind Image Quality Assessment by Learning-to-Rank Discriminable Image Pairs](https://mlsanity.com/paper/1904.06505/) (2019)
- [Perceptual Image Quality Assessment with Transformers](https://mlsanity.com/paper/2104.14730/) (2021)
- [GIQA: Generated Image Quality Assessment](https://mlsanity.com/paper/2003.08932/) (2020)
- [Hallucinated-IQA: No-Reference Image Quality Assessment via Adversarial Learning](https://mlsanity.com/paper/1804.01681/) (2018)
- [Learning Conditional Knowledge Distillation for Degraded-Reference Image Quality Assessment](https://mlsanity.com/paper/2108.07948/) (2021)
- [Learning to Rank for Blind Image Quality Assessment](https://mlsanity.com/paper/1309.0213/) (2013)
- [Active Fine-Tuning from gMAD Examples Improves Blind Image Quality Assessment](https://mlsanity.com/paper/2003.03849/) (2020)
- [Blind Predicting Similar Quality Map for Image Quality Assessment](https://mlsanity.com/paper/1805.08493/) (2018)
- [Region-Adaptive Deformable Network for Image Quality Assessment](https://mlsanity.com/paper/2104.11599/) (2021)
- [RAN4IQA: Restorative Adversarial Nets for No-Reference Image Quality Assessment](https://mlsanity.com/paper/1712.05444/) (2017)

## Built on by (most cited first, 1 of 1)

- [Multi-Scale Features and Parallel Transformers Based Image Quality Assessment](https://mlsanity.com/paper/2204.09779/) (2022)

## Abstract (arXiv)

Objective image quality evaluation is a challenging task, which aims to measure the quality of a given image automatically. According to the availability of the reference images, there are Full-Reference and No-Reference IQA tasks, respectively. Most deep learning approaches use regression from deep features extracted by Convolutional Neural Networks. For the FR task, another option is conducting a statistical comparison on deep features. For all these methods, non-local information is usually neglected. In addition, the relationship between FR and NR tasks is less explored. Motivated by the recent success of transformers in modeling contextual information, we propose a unified IQA framework that utilizes CNN backbone and transformer encoder to extract features. The proposed framework is compatible with both FR and NR modes and allows for a joint training scheme. Evaluation experiments on three standard IQA datasets, i.e., LIVE, CSIQ and TID2013, and KONIQ-10K, show that our proposed model can achieve state-of-the-art FR performance. In addition, comparable NR performance is achieved in extensive experiments, and the results show that the NR performance can be leveraged by the joint training scheme.