# You Only Train Once: A Unified Framework for Both Full-Reference and No-Reference Image Quality Assessment arXiv: https://arxiv.org/abs/2310.09560 ยท HTML: https://mlsanity.com/paper/2310.09560/ ## Key facts - Cited by 4 papers in the map (data through 14 Oct 2026). - Builds on 29 of its 81 references that are in the map. - First posted on arXiv 14 Oct 2023. ## Builds on (most cited first, 29 of 29) - [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017) - [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015) - [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014) - [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020) - [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019) - [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021) - [CrossViT: Cross-Attention Multi-Scale Vision Transformer for Image Classification](https://mlsanity.com/paper/2103.14899/) (2021) - [MANIQA: Multi-dimension Attention Network for No-Reference Image Quality Assessment](https://mlsanity.com/paper/2204.08958/) (2022) - [Stacked Cross Attention for Image-Text Matching](https://mlsanity.com/paper/1803.08024/) (2018) - [KonIQ-10k: An ecologically valid database for deep learning of blind image quality assessment](https://mlsanity.com/paper/1910.06180/) (2019) - [Blind Image Quality Assessment Using A Deep Bilinear Convolutional Neural Network](https://mlsanity.com/paper/1907.02665/) (2019) - [Deep Neural Networks for No-Reference and Full-Reference Image Quality Assessment](https://mlsanity.com/paper/1612.01697/) (2016) - [From Patches to Pictures (PaQ-2-PiQ): Mapping the Perceptual Space of Picture Quality](https://mlsanity.com/paper/1912.10088/) (2019) - [Massive Online Crowdsourced Study of Subjective and Objective Picture Quality](https://mlsanity.com/paper/1511.02919/) (2015) - [PIPAL: a Large-Scale Image Quality Assessment Dataset for Perceptual Image Restoration](https://mlsanity.com/paper/2007.12142/) (2020) - [No-Reference Image Quality Assessment via Transformers, Relative Ranking, and Self-Consistency](https://mlsanity.com/paper/2108.06858/) (2021) - [PieAPP: Perceptual Image-Error Assessment through Pairwise Preference](https://mlsanity.com/paper/1806.02067/) (2018) - [RankIQA: Learning from Rankings for No-reference Image Quality Assessment](https://mlsanity.com/paper/1707.08347/) (2017) - [MetaIQA: Deep Meta-learning for No-Reference Image Quality Assessment](https://mlsanity.com/paper/2004.05508/) (2020) - [Global Context-Aware Progressive Aggregation Network for Salient Object Detection](https://mlsanity.com/paper/2003.00651/) (2020) - [Image Quality Assessment using Contrastive Learning](https://mlsanity.com/paper/2110.13266/) (2021) - [CAT: Cross Attention in Vision Transformer](https://mlsanity.com/paper/2106.05786/) (2021) - [Transformer for Image Quality Assessment](https://mlsanity.com/paper/2101.01097/) (2020) - [Attentions Help CNNs See Better: Attention-based Hybrid Image Quality Assessment Network](https://mlsanity.com/paper/2204.10485/) (2022) - [Perceptual Image Quality Assessment with Transformers](https://mlsanity.com/paper/2104.14730/) (2021) - [Hallucinated-IQA: No-Reference Image Quality Assessment via Adversarial Learning](https://mlsanity.com/paper/1804.01681/) (2018) - [No-Reference Image Quality Assessment by Hallucinating Pristine Features](https://mlsanity.com/paper/2108.04165/) (2021) - [MSTRIQ: No Reference Image Quality Assessment Based on Swin Transformer with Multi-Stage Fusion](https://mlsanity.com/paper/2205.10101/) (2022) - [Quality Prediction on Deep Generative Images](https://mlsanity.com/paper/2004.08245/) (2020) ## Built on by (most cited first, 4 of 4) - [LAR-IQA: A Lightweight, Accurate, and Robust No-Reference Image Quality Assessment Model](https://mlsanity.com/paper/2408.17057/) (2024) - [Quality Prediction of AI Generated Images and Videos: Emerging Trends and Opportunities](https://mlsanity.com/paper/2410.08534/) (2024) - [MSLIQA: Enhancing Learning Representations for Image Quality Assessment through Multi-Scale Learning](https://mlsanity.com/paper/2408.16879/) (2024) - [Self-Supervised Image Super-Resolution Quality Assessment based on Content-Free Multi-Model Oriented Representation Learning](https://mlsanity.com/paper/2602.10744/) (2026) ## Abstract (arXiv) Although recent efforts in image quality assessment (IQA) have achieved promising performance, there still exists a considerable gap compared to the human visual system (HVS). One significant disparity lies in humans' seamless transition between full reference (FR) and no reference (NR) tasks, whereas existing models are constrained to either FR or NR tasks. This disparity implies the necessity of designing two distinct systems, thereby greatly diminishing the model's versatility. Therefore, our focus lies in unifying FR and NR IQA under a single framework. Specifically, we first employ an encoder to extract multi-level features from input images. Then a Hierarchical Attention (HA) module is proposed as a universal adapter for both FR and NR inputs to model the spatial distortion at each encoder stage. Furthermore, considering that different distortions contaminate encoder stages and damage image semantic meaning differently, a Semantic Distortion Aware (SDA) module is proposed to examine feature correlations between shallow and deep layers of the encoder. By adopting HA and SDA, the proposed network can effectively perform both FR and NR IQA. When our proposed model is independently trained on NR or FR IQA tasks, it outperforms existing models and achieves state-of-the-art performance. Moreover, when trained jointly on NR and FR IQA tasks, it further enhances the performance of NR IQA while achieving on-par performance in the state-of-the-art FR IQA. You only train once to perform both IQA tasks. Code will be released at: https://github.com/BarCodeReader/YOTO.