mlsanityData to 7 Oct 2026

The Unreasonable Effectiveness of Deep Features as a Perceptual Metric

arXiv 1801.03924 ↗First posted 11 Jan 2018cs.CV cs.GRcited by 11,029

Not on the map. Why?

Cited by 2,605 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

While it is nearly effortless for humans to quickly assess the perceptual similarity between two images, the underlying processes are thought to be quite complex. Despite this, the most widely used perceptual metrics today, such as PSNR and SSIM, are simple, shallow functions, and fail to account for many nuances of human perception. Recently, the deep learning community has found that features of the VGG network trained on ImageNet classification has been remarkably useful as a training loss for image synthesis. But how perceptual are these so-called "perceptual losses"? What elements are critical for their success? To answer these questions, we introduce a new dataset of human perceptual similarity judgments. We systematically evaluate deep features across different architectures and tasks and compare them with classic metrics. We find that deep features outperform all previous metrics by large margins on our dataset. More surprisingly, this result is not restricted to ImageNet-trained VGG features, but holds across different deep architectures and levels of supervision (supervised, self-supervised, or even unsupervised). Our results suggest that perceptual similarity is an emergent property shared across deep visual representations.

Key facts. Cited by 11,029 papers in the map (data through 7 Oct 2026). First posted on arXiv 11 Jan 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2020: 1; 2021: 688; 2022: 1041; 2023: 1600; 2024: 2347; 2025: 2815; 2026: 2537.

11,029 papers.

Sort:

By area:

  1. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  2. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
  3. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,550
  4. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,286
  5. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,526
  6. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,409
  7. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,231
  8. 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,198
  9. 2021Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fieldscited by 2,013
  10. 2021Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fieldscited by 1,957
  11. 2021SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equationscited by 1,903
  12. 2023Consistency Modelscited by 1,778
38 more
  1. 2021Plenoxels: Radiance Fields without Neural Networkscited by 1,607
  2. 2023Zero-1-to-3: Zero-shot One Image to 3D Objectcited by 1,483
  3. 2022RePaint: Inpainting using Denoising Diffusion Probabilistic Modelscited by 1,419
  4. 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,287
  5. 2022Imagic: Text-Based Real Image Editing with Diffusion Modelscited by 1,172
  6. 2022Null-text Inversion for Editing Real Images using Guided Diffusion Modelscited by 1,168
  7. 20234D Gaussian Splatting for Real-Time Dynamic Scene Renderingcited by 1,144
  8. 2022Multi-Concept Customization of Text-to-Image Diffusioncited by 1,099
  9. 2022MaskGIT: Masked Generative Image Transformercited by 1,082
  10. 2021MUSIQ: Multi-scale Image Quality Transformercited by 1,058
  11. 2024Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Predictioncited by 1,005
  12. 2023One-step Diffusion with Distribution Matching Distillationcited by 978
  13. 2022Exploring CLIP for Assessing the Look and Feel of Imagescited by 974
  14. 2021Direct Voxel Grid Optimization: Super-fast Convergence for Radiance Fields Reconstructioncited by 961
  15. 2021Blended Diffusion for Text-driven Editing of Natural Imagescited by 924
  16. 2023Human Preference Score v2: A Solid Benchmark for Evaluating Human Preferences of Text-to-Image Synthesiscited by 847
  17. 2023LRM: Large Reconstruction Model for Single Image to 3Dcited by 834
  18. 2023Objaverse-XL: A Universe of 10M+ 3D Objectscited by 767
  19. 2024Emu3: Next-Token Prediction is All You Needcited by 761
  20. 2024Autoregressive Model Beats Diffusion: Llama for Scalable Image Generationcited by 760
  21. 2024Structured 3D Latents for Scalable and Versatile 3D Generationcited by 742
  22. 2021Depth-supervised NeRF: Fewer Views and Faster Training for Freecited by 732
  23. 2023Dynamic 3D Gaussians: Tracking by Persistent Dynamic View Synthesiscited by 731
  24. 2021Resolution-robust Large Mask Inpainting with Fourier Convolutionscited by 730
  25. 2024LGM: Large Multi-View Gaussian Model for High-Resolution 3D Content Creationcited by 730
  26. 2022Block-NeRF: Scalable Large Scene Neural View Synthesiscited by 720
  27. 2021IBRNet: Learning Multi-View Image-Based Renderingcited by 698
  28. 2023Scaling Vision Transformers to 22 Billion Parameterscited by 690
  29. 2023Scaffold-GS: Structured 3D Gaussians for View-Adaptive Renderingcited by 684
  30. 2023Animate Anyone: Consistent and Controllable Image-to-Video Synthesis for Character Animationcited by 683
  31. 2023Wonder3D: Single Image to 3D using Cross-Domain Diffusioncited by 682
  32. 2021KiloNeRF: Speeding up Neural Radiance Fields with Thousands of Tiny MLPscited by 672
  33. 2023pixelSplat: 3D Gaussian Splats from Image Pairs for Scalable Generalizable 3D Reconstructioncited by 668
  34. 2021Vector-quantized Image Modeling with Improved VQGANcited by 656
  35. 2024Improved Distribution Matching Distillation for Fast Image Synthesiscited by 648
  36. 2021Common Objects in 3D: Large-Scale Learning and Evaluation of Real-life 3D Category Reconstructioncited by 646
  37. 2024Autoregressive Image Generation without Vector Quantizationcited by 644
  38. 2022DiffEdit: Diffusion-based semantic image editing with mask guidancecited by 643

50 most cited of 11,029 All 11,029 (JSON)