# Conformer and Blind Noisy Students for Improved Image Quality Assessment

arXiv: https://arxiv.org/abs/2204.12819 · HTML: https://mlsanity.com/paper/2204.12819/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 70 references that are in the map.
- First posted on arXiv 27 Apr 2022.

## Builds on (most cited first, 30 of 36)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [A ConvNet for the 2020s](https://mlsanity.com/paper/2201.03545/) (2022)
- [Taming Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2012.09841/) (2020)
- [Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network](https://mlsanity.com/paper/1609.04802/) (2016)
- [Conformer: Convolution-augmented Transformer for Speech Recognition](https://mlsanity.com/paper/2005.08100/) (2020)
- [Improved Regularization of Convolutional Neural Networks with Cutout](https://mlsanity.com/paper/1708.04552/) (2017)
- [Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning](https://mlsanity.com/paper/1602.07261/) (2016)
- [Self-training with Noisy Student improves ImageNet classification](https://mlsanity.com/paper/1911.04252/) (2019)
- [ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks](https://mlsanity.com/paper/1809.00219/) (2018)
- [Averaging Weights Leads to Wider Optima and Better Generalization](https://mlsanity.com/paper/1803.05407/) (2018)
- [Image Quality Assessment: Unifying Structure and Texture Similarity](https://mlsanity.com/paper/2004.07728/) (2020)
- [The Perception-Distortion Tradeoff](https://mlsanity.com/paper/1711.06077/) (2017)
- [KonIQ-10k: An ecologically valid database for deep learning of blind image quality assessment](https://mlsanity.com/paper/1910.06180/) (2019)
- [Deep Neural Networks for No-Reference and Full-Reference Image Quality Assessment](https://mlsanity.com/paper/1612.01697/) (2016)
- [SRFlow: Learning the Super-Resolution Space with Normalizing Flow](https://mlsanity.com/paper/2006.14200/) (2020)
- [Gradient Magnitude Similarity Deviation: A Highly Efficient Perceptual Image Quality Index](https://mlsanity.com/paper/1308.3052/) (2013)
- [PIPAL: a Large-Scale Image Quality Assessment Dataset for Perceptual Image Restoration](https://mlsanity.com/paper/2007.12142/) (2020)
- [PieAPP: Perceptual Image-Error Assessment through Pairwise Preference](https://mlsanity.com/paper/1806.02067/) (2018)
- [The 2018 PIRM Challenge on Perceptual Image Super-resolution](https://mlsanity.com/paper/1809.07517/) (2018)
- [Learning a No-Reference Quality Metric for Single-Image Super-Resolution](https://mlsanity.com/paper/1612.05890/) (2016)
- [Uncertainty-Aware Blind Image Quality Assessment in the Laboratory and Wild](https://mlsanity.com/paper/2005.13983/) (2020)
- [Perceptual Image Quality Assessment with Transformers](https://mlsanity.com/paper/2104.14730/) (2021)
- [Seven ways to improve example-based single image super resolution](https://mlsanity.com/paper/1511.02228/) (2015)
- [NTIRE 2021 Challenge on Image Deblurring](https://mlsanity.com/paper/2104.14854/) (2021)
- [Efficient conformer: Progressive downsampling and grouped attention for automatic speech recognition](https://mlsanity.com/paper/2109.01163/) (2021)
- [Image Quality Assessment for Perceptual Image Restoration: A New Dataset, Benchmark and Metric](https://mlsanity.com/paper/2011.15002/) (2020)

## Built on by (most cited first, 5 of 5)

- [Swin2SR: SwinV2 Transformer for Compressed Image Super-Resolution and Restoration](https://mlsanity.com/paper/2209.11345/) (2022)
- [NTIRE 2022 Challenge on Perceptual Image Quality Assessment](https://mlsanity.com/paper/2206.11695/) (2022)
- [Perceptual Image Enhancement for Smartphone Real-Time Applications](https://mlsanity.com/paper/2210.13552/) (2022)
- [Real-Time Under-Display Cameras Image Restoration and HDR on Mobile Devices](https://mlsanity.com/paper/2211.14040/) (2022)
- [Blind CT Image Quality Assessment Using DDPM-derived Content and Transformer-based Evaluator](https://mlsanity.com/paper/2310.03118/) (2023)

## Abstract (arXiv)

Generative models for image restoration, enhancement, and generation have significantly improved the quality of the generated images. Surprisingly, these models produce more pleasant images to the human eye than other methods, yet, they may get a lower perceptual quality score using traditional perceptual quality metrics such as PSNR or SSIM. Therefore, it is necessary to develop a quantitative metric to reflect the performance of new algorithms, which should be well-aligned with the person's mean opinion score (MOS). Learning-based approaches for perceptual image quality assessment (IQA) usually require both the distorted and reference image for measuring the perceptual quality accurately. However, commonly only the distorted or generated image is available. In this work, we explore the performance of transformer-based full-reference IQA models. We also propose a method for IQA based on semi-supervised knowledge distillation from full-reference teacher models into blind student models using noisy pseudo-labeled data. Our approaches achieved competitive results on the NTIRE 2022 Perceptual Image Quality Assessment Challenge: our full-reference model was ranked 4th, and our blind noisy student was ranked 3rd among 70 participants, each in their respective track.