# E-LPIPS: Robust Perceptual Image Similarity via Random Transformation Ensembles

arXiv: https://arxiv.org/abs/1906.03973 · HTML: https://mlsanity.com/paper/1906.03973/

## Key facts

- Cited by 45 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Jun 2019.

## Built on by (most cited first, 30 of 45)

- [Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fields](https://mlsanity.com/paper/2111.12077/) (2021)
- [DreamSim: Learning New Dimensions of Human Visual Similarity using Synthetic Data](https://mlsanity.com/paper/2306.09344/) (2023)
- [Dataset Distillation: A Comprehensive Review](https://mlsanity.com/paper/2301.07014/) (2023)
- [Deep Image Deblurring: A Survey](https://mlsanity.com/paper/2201.10700/) (2022)
- [Distilling Diffusion Models into Conditional GANs](https://mlsanity.com/paper/2405.05967/) (2024)
- [Shift-tolerant Perceptual Similarity Metric](https://mlsanity.com/paper/2207.13686/) (2022)
- [OutCast: Outdoor Single-image Relighting with Cast Shadows](https://mlsanity.com/paper/2204.09341/) (2022)
- [R-LPIPS: An Adversarially Robust Perceptual Similarity Metric](https://mlsanity.com/paper/2307.15157/) (2023)
- [Towards Benchmarking and Assessing Visual Naturalness of Physical World Adversarial Attacks](https://mlsanity.com/paper/2305.12863/) (2023)
- [Universal Perturbation Attack on Differentiable No-Reference Image- and Video-Quality Metrics](https://mlsanity.com/paper/2211.00366/) (2022)
- [ZeroComp: Zero-shot Object Compositing from Image Intrinsics via Diffusion](https://mlsanity.com/paper/2410.08168/) (2024)
- [LipSim: A Provably Robust Perceptual Similarity Metric](https://mlsanity.com/paper/2310.18274/) (2023)
- [Video Quality Assessment: A Comprehensive Survey](https://mlsanity.com/paper/2412.04508/) (2024)
- [Comparing the Robustness of Modern No-Reference Image- and Video-Quality Metrics to Adversarial Attacks](https://mlsanity.com/paper/2310.06958/) (2023)
- [Neural Relightable Participating Media Rendering](https://mlsanity.com/paper/2110.12993/) (2021)
- [A Systematic Performance Analysis of Deep Perceptual Loss Networks: Breaking Transfer Learning Conventions](https://mlsanity.com/paper/2302.04032/) (2023)
- [Attacking Perceptual Similarity Metrics](https://mlsanity.com/paper/2305.08840/) (2023)
- [Vi(E)va LLM! A Conceptual Stack for Evaluating and Interpreting Generative AI-based Visualizations](https://mlsanity.com/paper/2402.02167/) (2024)
- [Adversarially Robust CLIP Models Can Induce Better (Robust) Perceptual Metrics](https://mlsanity.com/paper/2502.11725/) (2025)
- [Improving Robustness of Reinforcement Learning for Power System Control with Adversarial Training](https://mlsanity.com/paper/2110.08956/) (2021)
- [Augmenting Perceptual Super-Resolution via Image Quality Predictors](https://mlsanity.com/paper/2504.18524/) (2025)
- [FreeStyleGAN: Free-view Editable Portrait Rendering with the Camera Manifold](https://mlsanity.com/paper/2109.09378/) (2021)
- [Sandwiched Video Compression: Efficiently Extending the Reach of Standard Codecs with Neural Wrappers](https://mlsanity.com/paper/2303.11473/) (2023)
- [Hallucination Score: Towards Mitigating Hallucinations in Generative Image Super-Resolution](https://mlsanity.com/paper/2507.14367/) (2025)
- [Adversarial purification for no-reference image-quality metrics: applicability study and new methods](https://mlsanity.com/paper/2404.06957/) (2024)
- [Identifying and Mitigating Flaws of Deep Perceptual Similarity Metrics](https://mlsanity.com/paper/2207.02512/) (2022)
- [Neural Style Transfer for Vector Graphics](https://mlsanity.com/paper/2303.03405/) (2023)
- [JoIN: Joint GANs Inversion for Intrinsic Image Decomposition](https://mlsanity.com/paper/2305.11321/) (2023)
- [Ti-Patch: Tiled Physical Adversarial Patch for no-reference video quality metrics](https://mlsanity.com/paper/2404.09961/) (2024)
- [Lagrangian Objective Function Leads to Improved Unforeseen Attack Generalization in Adversarial Training](https://mlsanity.com/paper/2103.15385/) (2021)

## Abstract (arXiv)

It has been recently shown that the hidden variables of convolutional neural networks make for an efficient perceptual similarity metric that accurately predicts human judgment on relative image similarity assessment. First, we show that such learned perceptual similarity metrics (LPIPS) are susceptible to adversarial attacks that dramatically contradict human visual similarity judgment. While this is not surprising in light of neural networks' well-known weakness to adversarial perturbations, we proceed to show that self-ensembling with an infinite family of random transformations of the input --- a technique known not to render classification networks robust --- is enough to turn the metric robust against attack, while retaining predictive power on human judgments. Finally, we study the geometry imposed by our our novel self-ensembled metric (E-LPIPS) on the space of natural images. We find evidence of "perceptual convexity" by showing that convex combinations of similar-looking images retain appearance, and that discrete geodesics yield meaningful frame interpolation and texture morphing, all without explicit correspondences.