# The Unreasonable Effectiveness of Linear Prediction as a Perceptual Metric

arXiv: https://arxiv.org/abs/2310.05986 · HTML: https://mlsanity.com/paper/2310.05986/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 27 references that are in the map.
- First posted on arXiv 6 Oct 2023.

## Builds on (most cited first, 9 of 9)

- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Variational image compression with a scale hyperprior](https://mlsanity.com/paper/1802.01436/) (2018)
- [End-to-end Optimized Image Compression](https://mlsanity.com/paper/1611.01704/) (2016)
- [Image Quality Assessment using Contrastive Learning](https://mlsanity.com/paper/2110.13266/) (2021)
- [Do better ImageNet classifiers assess perceptual similarity better?](https://mlsanity.com/paper/2203.04946/) (2022)
- [An Unsupervised Information-Theoretic Perceptual Quality Metric](https://mlsanity.com/paper/2006.06752/) (2020)
- [Quantifying Visual Image Quality: A Bayesian View](https://mlsanity.com/paper/2102.00195/) (2021)
- [CONVIQT: Contrastive Video Quality Estimator](https://mlsanity.com/paper/2206.14713/) (2022)

## Built on by (most cited first, 3 of 3)

- [ChatVis: Large Language Model Agent for Generating Scientific Visualizations](https://mlsanity.com/paper/2507.23096/) (2025)
- [On the Utility of Foundation Models for Fast MRI: Vision-Language-Guided Image Reconstruction](https://mlsanity.com/paper/2511.19641/) (2025)
- [JLD: Perceptual Distance Through A Jacobian Lens](https://mlsanity.com/paper/2610.05967/) (2026)

## Abstract (arXiv)

We show how perceptual embeddings of the visual system can be constructed at inference-time with no training data or deep neural network features. Our perceptual embeddings are solutions to a weighted least squares (WLS) problem, defined at the pixel-level, and solved at inference-time, that can capture global and local image characteristics. The distance in embedding space is used to define a perceptual similarity metric which we call LASI: Linear Autoregressive Similarity Index. Experiments on full-reference image quality assessment datasets show LASI performs competitively with learned deep feature based methods like LPIPS (Zhang et al., 2018) and PIM (Bhardwaj et al., 2020), at a similar computational cost to hand-crafted methods such as MS-SSIM (Wang et al., 2003). We found that increasing the dimensionality of the embedding space consistently reduces the WLS loss while increasing performance on perceptual tasks, at the cost of increasing the computational complexity. LASI is fully differentiable, scales cubically with the number of embedding dimensions, and can be parallelized at the pixel-level. A Maximum Differentiation (MAD) competition (Wang & Simoncelli, 2008) between LASI and LPIPS shows that both methods are capable of finding failure points for the other, suggesting these metrics can be combined.