# ColorVideoVDP: A visual difference predictor for image, video and display distortions

arXiv: https://arxiv.org/abs/2401.11485 · HTML: https://mlsanity.com/paper/2401.11485/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 79 references that are in the map.
- First posted on arXiv 21 Jan 2024.

## Builds on (most cited first, 10 of 10)

- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [Training Deep Nets with Sublinear Memory Cost](https://mlsanity.com/paper/1604.06174/) (2016)
- [PieAPP: Perceptual Image-Error Assessment through Pairwise Preference](https://mlsanity.com/paper/1806.02067/) (2018)
- [Perceptual Image Quality Assessment with Transformers](https://mlsanity.com/paper/2104.14730/) (2021)
- [HDR-VDP-3: A multi-metric for predicting image differences, quality and contrast distortions in high dynamic range and regular content](https://mlsanity.com/paper/2304.13625/) (2023)
- [Active Sampling for Pairwise Comparisons via Approximate Message Passing and Information Gain Maximization](https://mlsanity.com/paper/2004.05691/) (2020)
- [Subjective Assessment of High Dynamic Range Videos Under Different Ambient Conditions](https://mlsanity.com/paper/2209.10005/) (2022)
- [Consolidated Dataset and Metrics for High-Dynamic-Range Image Quality](https://mlsanity.com/paper/2012.10758/) (2020)
- [FUNQUE: Fusion of Unified Quality Evaluators](https://mlsanity.com/paper/2202.11241/) (2022)

## Built on by (most cited first, 30 of 34)

- [DiffusionRenderer: Neural Inverse and Forward Rendering with Video Diffusion Models](https://mlsanity.com/paper/2501.18590/) (2025)
- [Video Seal: Open and Efficient Video Watermarking](https://mlsanity.com/paper/2412.09492/) (2024)
- [Pixel Seal: Adversarial-only training for invisible image and video watermarking](https://mlsanity.com/paper/2512.16874/) (2025)
- [Do computer vision foundation models learn the low-level characteristics of the human visual system?](https://mlsanity.com/paper/2502.20256/) (2025)
- [X2HDR: HDR Image Generation in a Perceptually Uniform Space](https://mlsanity.com/paper/2602.04814/) (2026)
- [Puzzle Similarity: A Perceptually-guided Cross-Reference Metric for Artifact Detection in 3D Scene Reconstructions](https://mlsanity.com/paper/2411.17489/) (2024)
- [Subjective Visual Quality Assessment for High-Fidelity Learning-Based Image Compression](https://mlsanity.com/paper/2504.06301/) (2025)
- [Capturing Stable HDR Videos Using a Dual-Camera System](https://mlsanity.com/paper/2507.06593/) (2025)
- [Evaluating quality metrics through the lenses of psychophysical measurements of low-level vision](https://mlsanity.com/paper/2503.16264/) (2025)
- [HDR Video Generation via Latent Alignment with Logarithmic Encoding](https://mlsanity.com/paper/2604.11788/) (2026)
- [Deep chroma compression of tone-mapped images](https://mlsanity.com/paper/2409.16032/) (2024)
- [Fine-Grained HDR Image Quality Assessment From Noticeably Distorted to Very High Fidelity](https://mlsanity.com/paper/2506.12505/) (2025)
- [ICME 2025 Generalizable HDR and SDR Video Quality Measurement Grand Challenge](https://mlsanity.com/paper/2506.22790/) (2025)
- [CameraVDP: Perceptual Display Assessment with Uncertainty Estimation via Camera and Visual Difference Prediction](https://mlsanity.com/paper/2509.08947/) (2025)
- [D-PerceptCT: Deep Perceptual Enhancement for Low-Dose CT Images](https://mlsanity.com/paper/2511.14518/) (2025)
- [How Accurate are Video Quality Models for Diffusion-Based Video Super-Resolution?](https://mlsanity.com/paper/2605.25940/) (2026)
- [Human Vision Constrained Super-Resolution](https://mlsanity.com/paper/2411.17513/) (2024)
- [Collaborative Texture Filtering](https://mlsanity.com/paper/2506.17770/) (2025)
- [LCS: An AI-based Low-Complexity Scaler for Power-Efficient Super-Resolution of Game Content](https://mlsanity.com/paper/2507.22873/) (2025)
- [FDIM: A Feature-distance-based Generic Video Quality Metric for Versatile Codecs](https://mlsanity.com/paper/2604.24123/) (2026)
- [Velox: Learning Representations of 4D Geometry and Appearance](https://mlsanity.com/paper/2605.04527/) (2026)
- [Streaming of rendered content with adaptive frame rate and resolution](https://mlsanity.com/paper/2605.10995/) (2026)
- [Generating HDR Video from SDR Video](https://mlsanity.com/paper/2605.14703/) (2026)
- [Towards Standardized Light Field Quality Assessment: Hybrid Subjective Benchmarking and Objective Metric Evaluation](https://mlsanity.com/paper/2607.03494/) (2026)
- [Learning Flexible Generalization in Video Quality Assessment by Bringing Device and Viewing Condition Distributions](https://mlsanity.com/paper/2607.04643/) (2026)
- [JPEG AIC2026: A large-scale dataset for fine-grained assessment of image coding](https://mlsanity.com/paper/2607.22783/) (2026)
- [Hologram Representation via Quadratic Phase Gaussian Splatting](https://mlsanity.com/paper/2609.11434/) (2026)
- [Recurrent Dynamic Range Extension](https://mlsanity.com/paper/2609.13135/) (2026)
- [An Efficient and Effective Watermarking Scheme for the Protection of the Intellectual Property Rights of Video Generative Models](https://mlsanity.com/paper/2609.23586/) (2026)
- [Ultra-fast Neural Inference for Stochastic Gaussian Splatting Denoising](https://mlsanity.com/paper/2609.25604/) (2026)

## Abstract (arXiv)

ColorVideoVDP is a video and image quality metric that models spatial and temporal aspects of vision, for both luminance and color. The metric is built on novel psychophysical models of chromatic spatiotemporal contrast sensitivity and cross-channel contrast masking. It accounts for the viewing conditions, geometric, and photometric characteristics of the display. It was trained to predict common video streaming distortions (e.g. video compression, rescaling, and transmission errors), and also 8 new distortion types related to AR/VR displays (e.g. light source and waveguide non-uniformities). To address the latter application, we collected our novel XR-Display-Artifact-Video quality dataset (XR-DAVID), comprised of 336 distorted videos. Extensive testing on XR-DAVID, as well as several datasets from the literature, indicate a significant gain in prediction performance compared to existing metrics. ColorVideoVDP opens the doors to many novel applications which require the joint automated spatiotemporal assessment of luminance and color distortions, including video streaming, display specification and design, visual comparison of results, and perceptually-guided quality optimization.