# On the relation between statistical learning and perceptual distances

arXiv: https://arxiv.org/abs/2106.04427 · HTML: https://mlsanity.com/paper/2106.04427/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 61 references that are in the map.
- First posted on arXiv 8 Jun 2021.

## Builds on (most cited first, 19 of 19)

- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Variational image compression with a scale hyperprior](https://mlsanity.com/paper/1802.01436/) (2018)
- [Image Quality Assessment: Unifying Structure and Texture Similarity](https://mlsanity.com/paper/2004.07728/) (2020)
- [End-to-end Optimized Image Compression](https://mlsanity.com/paper/1611.01704/) (2016)
- [PixelCNN++: Improving the PixelCNN with Discretized Logistic Mixture Likelihood and Other Modifications](https://mlsanity.com/paper/1701.05517/) (2017)
- [Generative Adversarial Networks for Extreme Learned Image Compression](https://mlsanity.com/paper/1804.02958/) (2018)
- [Density Modeling of Images using a Generalized Normalization Transformation](https://mlsanity.com/paper/1511.06281/) (2015)
- [Understanding and Improving Interpolation in Autoencoders via an Adversarial Regularizer](https://mlsanity.com/paper/1807.07543/) (2018)
- [Deep Neural Networks Rival the Representation of Primate IT Cortex for Core Visual Object Recognition](https://mlsanity.com/paper/1406.3284/) (2014)
- [An Unsupervised Information-Theoretic Perceptual Quality Metric](https://mlsanity.com/paper/2006.06752/) (2020)
- [Eigen-Distortions of Hierarchical Representations](https://mlsanity.com/paper/1710.02266/) (2017)
- [Representing Closed Transformation Paths in Encoded Network Latent Space](https://mlsanity.com/paper/1912.02644/) (2019)
- [Texture Interpolation for Probing Visual Perception](https://mlsanity.com/paper/2006.03698/) (2020)
- [Contrast Sensitivity Functions in Autoencoders](https://mlsanity.com/paper/2103.00481/) (2021)
- [Nonlinearities and Adaptation of Color Vision from Sequential Principal Curves Analysis](https://mlsanity.com/paper/1602.00236/) (2016)
- [Derivatives and Inverse of Cascaded Linear+Nonlinear Neural Models](https://mlsanity.com/paper/1711.00526/) (2017)
- [Enforcing Perceptual Consistency on Generative Adversarial Networks by Using the Normalised Laplacian Pyramid Distance](https://mlsanity.com/paper/1908.04347/) (2019)
- [Spatio-Chromatic Information available from different Neural Layers via Gaussianization](https://mlsanity.com/paper/1910.01559/) (2019)
- [Visual Information flow in Wilson-Cowan networks](https://mlsanity.com/paper/1907.13046/) (2019)

## Built on by (most cited first, 12 of 12)

- [Looks Too Good To Be True: An Information-Theoretic Analysis of Hallucinations in Generative Restoration Models](https://mlsanity.com/paper/2405.16475/) (2024)
- [Neural Image Compression: Generalization, Robustness, and Spectral Biases](https://mlsanity.com/paper/2307.08657/) (2023)
- [Microdosing: Knowledge Distillation for GAN based Compression](https://mlsanity.com/paper/2201.02624/) (2022)
- [Perceptual Scales Predicted by Fisher Information Metrics](https://mlsanity.com/paper/2310.11759/) (2023)
- [Assessing invariance to affine transformations in image quality metrics](https://mlsanity.com/paper/2407.17927/) (2024)
- [Sampling Based On Natural Image Statistics Improves Local Surrogate Explainers](https://mlsanity.com/paper/2208.03961/) (2022)
- [Image Statistics Predict the Sensitivity of Perceptual Quality Metrics](https://mlsanity.com/paper/2303.09874/) (2023)
- [Data is Overrated: Perceptual Metrics Can Lead Learning in the Absence of Training Data](https://mlsanity.com/paper/2312.03455/) (2023)
- [Analysis of Deep Image Quality Models](https://mlsanity.com/paper/2302.13345/) (2023)
- [StableSemantics: A Synthetic Language-Vision Dataset of Semantic Representations in Naturalistic Images](https://mlsanity.com/paper/2406.13735/) (2024)
- [The Effect of Perceptual Metrics on Music Representation Learning for Genre Classification](https://mlsanity.com/paper/2409.17069/) (2024)
- [From Images to Perception: Emergence of Perceptual Properties by Reconstructing Images](https://mlsanity.com/paper/2508.10450/) (2025)

## Abstract (arXiv)

It has been demonstrated many times that the behavior of the human visual system is connected to the statistics of natural images. Since machine learning relies on the statistics of training data as well, the above connection has interesting implications when using perceptual distances (which mimic the behavior of the human visual system) as a loss function. In this paper, we aim to unravel the non-trivial relationships between the probability distribution of the data, perceptual distances, and unsupervised machine learning. To this end, we show that perceptual sensitivity is correlated with the probability of an image in its close neighborhood. We also explore the relation between distances induced by autoencoders and the probability distribution of the training data, as well as how these induced distances are correlated with human perception. Finally, we find perceptual distances do not always lead to noticeable gains in performance over Euclidean distance in common image processing tasks, except when data is scarce and the perceptual distance provides regularization. We propose this may be due to a double-counting effect of the image statistics, once in the perceptual distance and once in the training procedure.