# Generating Images with Perceptual Similarity Metrics based on Deep Networks

arXiv: https://arxiv.org/abs/1602.02644 · HTML: https://mlsanity.com/paper/1602.02644/

## Key facts

- Cited by 222 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Feb 2016.

## Built on by (most cited first, 30 of 222)

- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Image Super-Resolution via Iterative Refinement](https://mlsanity.com/paper/2104.07636/) (2021)
- [Palette: Image-to-Image Diffusion Models](https://mlsanity.com/paper/2111.05826/) (2021)
- [Common Objects in 3D: Large-Scale Learning and Evaluation of Real-life 3D Category Reconstruction](https://mlsanity.com/paper/2109.00512/) (2021)
- [DreamSim: Learning New Dimensions of Human Visual Similarity using Synthetic Data](https://mlsanity.com/paper/2306.09344/) (2023)
- [Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Models](https://mlsanity.com/paper/2103.04922/) (2021)
- [Bridging the Gap to Real-World Object-Centric Learning](https://mlsanity.com/paper/2209.14860/) (2022)
- [Projected GANs Converge Faster](https://mlsanity.com/paper/2111.01007/) (2021)
- [PeCo: Perceptual Codebook for BERT Pre-training of Vision Transformers](https://mlsanity.com/paper/2111.12710/) (2021)
- [Generative Diffusion Prior for Unified Image Restoration and Enhancement](https://mlsanity.com/paper/2304.01247/) (2023)
- [Lossy Image Compression with Conditional Diffusion Models](https://mlsanity.com/paper/2209.06950/) (2022)
- [EfficientAD: Accurate Visual Anomaly Detection at Millisecond-Level Latencies](https://mlsanity.com/paper/2303.14535/) (2023)
- [FILM: Frame Interpolation for Large Motion](https://mlsanity.com/paper/2202.04901/) (2022)
- [ImageBART: Bidirectional Context with Multinomial Diffusion for Autoregressive Image Synthesis](https://mlsanity.com/paper/2108.08827/) (2021)
- [Pixel-wise Anomaly Detection in Complex Driving Scenes](https://mlsanity.com/paper/2103.05445/) (2021)
- [Diffusion Models for Image Restoration and Enhancement: A Comprehensive Survey](https://mlsanity.com/paper/2308.09388/) (2023)
- [DiffuseVAE: Efficient, Controllable and High-Fidelity Generation from Low-Dimensional Latents](https://mlsanity.com/paper/2201.00308/) (2022)
- [Wonderland: Navigating 3D Scenes from a Single Image](https://mlsanity.com/paper/2412.12091/) (2024)
- [Masked Autoencoders Are Effective Tokenizers for Diffusion Models](https://mlsanity.com/paper/2502.03444/) (2025)
- [MoFaNeRF: Morphable Facial Neural Radiance Field](https://mlsanity.com/paper/2112.02308/) (2021)
- [Make It Move: Controllable Image-to-Video Generation with Text Descriptions](https://mlsanity.com/paper/2112.02815/) (2021)
- [Ensembling Off-the-shelf Models for GAN Training](https://mlsanity.com/paper/2112.09130/) (2021)
- [An Introduction to Neural Data Compression](https://mlsanity.com/paper/2202.06533/) (2022)
- [Deep Clustering: A Comprehensive Survey](https://mlsanity.com/paper/2210.04142/) (2022)
- [Distilling Diffusion Models into Conditional GANs](https://mlsanity.com/paper/2405.05967/) (2024)
- [VideoGen: A Reference-Guided Latent Diffusion Approach for High Definition Text-to-Video Generation](https://mlsanity.com/paper/2309.00398/) (2023)
- [Benchmarking Autoregressive Conditional Diffusion Models for Turbulent Flow Simulation](https://mlsanity.com/paper/2309.01745/) (2023)
- [Towards Realistic Scene Generation with LiDAR Diffusion Models](https://mlsanity.com/paper/2404.00815/) (2024)
- [The Spatially-Correlative Loss for Various Image Translation Tasks](https://mlsanity.com/paper/2104.00854/) (2021)
- [High Fidelity Visualization of What Your Self-Supervised Representation Knows About](https://mlsanity.com/paper/2112.09164/) (2021)

## Abstract (arXiv)

Image-generating machine learning models are typically trained with loss functions based on distance in the image space. This often leads to over-smoothed results. We propose a class of loss functions, which we call deep perceptual similarity metrics (DeePSiM), that mitigate this problem. Instead of computing distances in the image space, we compute distances between image features extracted by deep neural networks. This metric better reflects perceptually similarity of images and thus leads to better results. We show three applications: autoencoder training, a modification of a variational autoencoder, and inversion of deep convolutional networks. In all cases, the generated images look sharp and resemble natural images.