# Evaluating quality metrics through the lenses of psychophysical measurements of low-level vision

arXiv: https://arxiv.org/abs/2503.16264 · HTML: https://mlsanity.com/paper/2503.16264/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 51 references that are in the map.
- First posted on arXiv 20 Mar 2025.

## Builds on (most cited first, 17 of 17)

- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Image Quality Assessment: Unifying Structure and Texture Similarity](https://mlsanity.com/paper/2004.07728/) (2020)
- [Deep Learning for Image Super-resolution: A Survey](https://mlsanity.com/paper/1902.06068/) (2019)
- [TOPIQ: A Top-down Approach from Semantics to Distortions for Image Quality Assessment](https://mlsanity.com/paper/2308.03060/) (2023)
- [Deep Neural Networks for No-Reference and Full-Reference Image Quality Assessment](https://mlsanity.com/paper/1612.01697/) (2016)
- [Gradient Magnitude Similarity Deviation: A Highly Efficient Perceptual Image Quality Index](https://mlsanity.com/paper/1308.3052/) (2013)
- [A Haar Wavelet-Based Perceptual Similarity Index for Image Quality Assessment](https://mlsanity.com/paper/1607.06140/) (2016)
- [Attentions Help CNNs See Better: Attention-based Hybrid Image Quality Assessment Network](https://mlsanity.com/paper/2204.10485/) (2022)
- [Shift-tolerant Perceptual Similarity Metric](https://mlsanity.com/paper/2207.13686/) (2022)
- [HDR-VDP-3: A multi-metric for predicting image differences, quality and contrast distortions in high dynamic range and regular content](https://mlsanity.com/paper/2304.13625/) (2023)
- [ColorVideoVDP: A visual difference predictor for image, video and display distortions](https://mlsanity.com/paper/2401.11485/) (2024)
- [Mean Deviation Similarity Index: Efficient and Reliable Full-Reference Image Quality Evaluator](https://mlsanity.com/paper/1608.07433/) (2016)
- [Multiscale Sliced Wasserstein Distances as Perceptual Color Difference Measures](https://mlsanity.com/paper/2407.10181/) (2024)
- [Do computer vision foundation models learn the low-level characteristics of the human visual system?](https://mlsanity.com/paper/2502.20256/) (2025)
- [A Survey on Perceptually Optimized Video Coding](https://mlsanity.com/paper/2112.12284/) (2021)
- [FUNQUE: Fusion of Unified Quality Evaluators](https://mlsanity.com/paper/2202.11241/) (2022)
- [Beyond Correlation: Evaluating Multimedia Quality Models with the Constrained Concordance Index](https://mlsanity.com/paper/2411.05794/) (2024)

## Built on by (most cited first, 2 of 2)

- [Modeling Beyond MOS: Quality Assessment Models Must Integrate Context, Reasoning, and Multimodality](https://mlsanity.com/paper/2505.19696/) (2025)
- [JPEG AIC2026: A large-scale dataset for fine-grained assessment of image coding](https://mlsanity.com/paper/2607.22783/) (2026)

## Abstract (arXiv)

Image and video quality metrics, such as SSIM, LPIPS, and VMAF, aim to predict perceived visual quality and are often assumed to reflect principles of human vision. However, relatively few metrics explicitly incorporate models of human perception, with most relying on hand-crafted formulas or data-driven training to approximate perceptual alignment. In this paper, we introduce a set of tests for full-reference quality metrics that evaluate their ability to capture key aspects of low-level human vision: contrast sensitivity, contrast masking, and contrast matching. These tests provide an additional framework for assessing both established and newly proposed metrics. We apply the tests to 34 existing quality metrics and highlight patterns in their behavior, including the ability of LPIPS and MS-SSIM to predict contrast masking and the tendency of SSIM to overemphasize high spatial frequencies, which is mitigated in MS-SSIM, and the general inability of metrics to model supra-threshold contrast constancy. Our results demonstrate how these tests can reveal properties of quality metrics that are not easily observed with standard evaluation protocols.