# Subjective Image Quality Assessment with Boosted Triplet Comparisons

arXiv: https://arxiv.org/abs/2108.00201 · HTML: https://mlsanity.com/paper/2108.00201/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 110 references that are in the map.
- First posted on arXiv 31 Jul 2021.

## Builds on (most cited first, 5 of 5)

- [VideoSet: A Large-Scale Compressed Video Quality Dataset Based on JND Measurement](https://mlsanity.com/paper/1701.01500/) (2017)
- [A practical guide and software for analysing pairwise comparison experiments](https://mlsanity.com/paper/1712.03686/) (2017)
- [Hybrid-MST: A Hybrid Active Sampling Strategy for Pairwise Preference Aggregation](https://mlsanity.com/paper/1810.08851/) (2018)
- [Estimation of perceptual scales using ordinal embedding](https://mlsanity.com/paper/1908.07962/) (2019)
- [Subjective Annotation for a Frame Interpolation Benchmark using Artefact Amplification](https://mlsanity.com/paper/2001.06409/) (2020)

## Built on by (most cited first, 9 of 9)

- [UHD-IQA Benchmark Database: Pushing the Boundaries of Blind Photo Quality Assessment](https://mlsanity.com/paper/2406.17472/) (2024)
- [Image Quality Assessment: Integrating Model-Centric and Data-Centric Approaches](https://mlsanity.com/paper/2207.14769/) (2022)
- [Relaxed forced choice improves performance of visual quality assessment methods](https://mlsanity.com/paper/2305.00220/) (2023)
- [Fine-grained subjective visual quality assessment for high-fidelity compressed images](https://mlsanity.com/paper/2410.09501/) (2024)
- [Fine-Grained HDR Image Quality Assessment From Noticeably Distorted to Very High Fidelity](https://mlsanity.com/paper/2506.12505/) (2025)
- [Improving Maximum Likelihood Difference Scaling method to measure inter content scale](https://mlsanity.com/paper/2203.13186/) (2022)
- [Towards Standardized Light Field Quality Assessment: Hybrid Subjective Benchmarking and Objective Metric Evaluation](https://mlsanity.com/paper/2607.03494/) (2026)
- [JPEG AIC2026: A large-scale dataset for fine-grained assessment of image coding](https://mlsanity.com/paper/2607.22783/) (2026)
- [Local SVD-Entropy Maps as a Complementary Structural Representation for Full-Reference and No-Reference Image Quality Assessment](https://mlsanity.com/paper/2609.27959/) (2026)

## Abstract (arXiv)

In subjective full-reference image quality assessment, differences between perceptual image qualities of the reference image and its distorted versions are evaluated, often using degradation category ratings (DCR). However, the DCR has been criticized since differences between rating categories on this ordinal scale might not be perceptually equidistant, and observers may have different understandings of the categories. Pair comparisons (PC) of distorted images, followed by Thurstonian reconstruction of scale values, overcome these problems. In addition, PC is more sensitive than DCR, and it can provide scale values in fractional, just noticeable difference (JND) units that express a precise perceptional interpretation. Still, the comparison of images of nearly the same quality can be difficult. We introduce boosting techniques embedded in more general triplet comparisons (TC) that increase the sensitivity even more. Boosting amplifies the artefacts of distorted images, enlarges their visual representation by zooming, increases the visibility of the distortions by a flickering effect, or combines some of the above. Experimental results show the effectiveness of boosted TC for seven types of distortion. We crowdsourced over 1.7 million responses to triplet questions. A detailed analysis shows that boosting increases the discriminatory power and allows to reduce the number of subjective ratings without sacrificing the accuracy of the resulting relative image quality values. Our technique paves the way to fine-grained image quality datasets, allowing for more distortion levels, yet with high-quality subjective annotations. We also provide the details for Thurstonian scale reconstruction from TC and our annotated dataset, KonFiG-IQA, containing 10 source images, processed using 7 distortion types at 12 or even 30 levels, uniformly spaced over a span of 3 JND units.