# A Loss Function for Generative Neural Networks Based on Watson's Perceptual Model

arXiv: https://arxiv.org/abs/2006.15057 · HTML: https://mlsanity.com/paper/2006.15057/

## Key facts

- Cited by 39 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Jun 2020.

## Built on by (most cited first, 30 of 39)

- [The Stable Signature: Rooting Watermarks in Latent Diffusion Models](https://mlsanity.com/paper/2303.15435/) (2023)
- [WAVES: Benchmarking the Robustness of Image Watermarks](https://mlsanity.com/paper/2401.08573/) (2024)
- [Investigating Prompt Engineering in Diffusion Models](https://mlsanity.com/paper/2211.15462/) (2022)
- [Video Seal: Open and Efficient Video Watermarking](https://mlsanity.com/paper/2412.09492/) (2024)
- [Shift-tolerant Perceptual Similarity Metric](https://mlsanity.com/paper/2207.13686/) (2022)
- [Attack-Resilient Image Watermarking Using Stable Diffusion](https://mlsanity.com/paper/2401.04247/) (2024)
- [Explicitly Minimizing the Blur Error of Variational Autoencoders](https://mlsanity.com/paper/2304.05939/) (2023)
- [WMAdapter: Adding WaterMark Control to Latent Diffusion Models](https://mlsanity.com/paper/2406.08337/) (2024)
- [WateRF: Robust Watermarks in Radiance Fields for Protection of Copyrights](https://mlsanity.com/paper/2405.02066/) (2024)
- [UnMarker: A Universal Attack on Defensive Image Watermarking](https://mlsanity.com/paper/2405.08363/) (2024)
- [Stable Signature is Unstable: Removing Image Watermark from Diffusion Models](https://mlsanity.com/paper/2405.07145/) (2024)
- [Boosting Latent Diffusion with Perceptual Objectives](https://mlsanity.com/paper/2411.04873/) (2024)
- [Evaluating the Interpretability of Generative Models by Interactive Reconstruction](https://mlsanity.com/paper/2102.01264/) (2021)
- [Clean Images are Hard to Reblur: Exploiting the Ill-Posed Inverse Task for Dynamic Scene Deblurring](https://mlsanity.com/paper/2104.12665/) (2021)
- [A Perceptual Quality Metric for Video Frame Interpolation](https://mlsanity.com/paper/2210.01879/) (2022)
- [Pixel Seal: Adversarial-only training for invisible image and video watermarking](https://mlsanity.com/paper/2512.16874/) (2025)
- [LAVIB: A Large-scale Video Interpolation Benchmark](https://mlsanity.com/paper/2406.09754/) (2024)
- [Attacking Perceptual Similarity Metrics](https://mlsanity.com/paper/2305.08840/) (2023)
- [Semantic similarity metrics for learned image registration](https://mlsanity.com/paper/2104.10051/) (2021)
- [Distributional Learning of Variational AutoEncoder: Application to Synthetic Data Generation](https://mlsanity.com/paper/2302.11294/) (2023)
- [FrePolad: Frequency-Rectified Point Latent Diffusion for Point Cloud Generation](https://mlsanity.com/paper/2311.12090/) (2023)
- [LVMark: Robust Watermark for Latent Video Diffusion Models](https://mlsanity.com/paper/2412.09122/) (2024)
- [Simpler is better: spectral regularization and up-sampling techniques for variational autoencoders](https://mlsanity.com/paper/2201.07544/) (2022)
- [Video Signature: Implicit Watermarking for Video Diffusion Models](https://mlsanity.com/paper/2506.00652/) (2025)
- [Towards Effective User Attribution for Latent Diffusion Models via Watermark-Informed Blending](https://mlsanity.com/paper/2409.10958/) (2024)
- [AesFA: An Aesthetic Feature-Aware Arbitrary Neural Style Transfer](https://mlsanity.com/paper/2312.05928/) (2023)
- [Enhancing image quality prediction with self-supervised visual masking](https://mlsanity.com/paper/2305.19858/) (2023)
- [CSIM: A Copula-based similarity index sensitive to local changes for Image quality assessment](https://mlsanity.com/paper/2410.01411/) (2024)
- [An Efficient Watermarking Method for Latent Diffusion Models via Low-Rank Adaptation and Dynamic Loss Weighting](https://mlsanity.com/paper/2410.20202/) (2024)
- [Pathology-Aware Adaptive Watermarking for Text-Driven Medical Image Synthesis](https://mlsanity.com/paper/2503.08346/) (2025)

## Abstract (arXiv)

To train Variational Autoencoders (VAEs) to generate realistic imagery requires a loss function that reflects human perception of image similarity. We propose such a loss function based on Watson's perceptual model, which computes a weighted distance in frequency space and accounts for luminance and contrast masking. We extend the model to color images, increase its robustness to translation by using the Fourier Transform, remove artifacts due to splitting the image into blocks, and make it differentiable. In experiments, VAEs trained with the new loss function generated realistic, high-quality image samples. Compared to using the Euclidean distance and the Structural Similarity Index, the images were less blurry; compared to deep neural network based losses, the new approach required less computational resources and generated images with less artifacts.