# Improving Image Autoencoder Embeddings with Perceptual Loss

arXiv: https://arxiv.org/abs/2001.03444 · HTML: https://mlsanity.com/paper/2001.03444/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Jan 2020.

## Built on by (most cited first, 13 of 13)

- [Perceptual Loss for Robust Unsupervised Homography Estimation](https://mlsanity.com/paper/2104.10011/) (2021)
- [A Systematic Performance Analysis of Deep Perceptual Loss Networks: Breaking Transfer Learning Conventions](https://mlsanity.com/paper/2302.04032/) (2023)
- [Foundational World Models Accurately Detect Bimanual Manipulator Failures](https://mlsanity.com/paper/2603.06987/) (2026)
- [A Critical Study on the Recent Deep Learning Based Semi-Supervised Video Anomaly Detection Methods](https://mlsanity.com/paper/2111.01604/) (2021)
- [A multimodal dynamical variational autoencoder for audiovisual speech representation learning](https://mlsanity.com/paper/2305.03582/) (2023)
- [H-GAN: the power of GANs in your Hands](https://mlsanity.com/paper/2103.15017/) (2021)
- [Exploiting generative self-supervised learning for the assessment of biological images with lack of annotations: a COVID-19 case-study](https://mlsanity.com/paper/2107.07761/) (2021)
- [Semantics and Spatiality of Emergent Communication](https://mlsanity.com/paper/2411.10173/) (2024)
- [Unsupervised End-to-End Training with a Self-Defined Target](https://mlsanity.com/paper/2403.12116/) (2024)
- [The Describe-Then-Generate Bottleneck: How VLM Descriptions Alter Image Generation Outcomes](https://mlsanity.com/paper/2509.18179/) (2025)
- [Advances in Electron Microscopy with Deep Learning](https://mlsanity.com/paper/2101.01178/) (2021)
- [DISentangled Counterfactual Visual interpretER (DISCOVER) generalizes to natural images](https://mlsanity.com/paper/2406.15918/) (2024)
- [Time series generation with spectrally aligned latent flow matching](https://mlsanity.com/paper/2609.21989/) (2026)

## Abstract (arXiv)

Autoencoders are commonly trained using element-wise loss. However, element-wise loss disregards high-level structures in the image which can lead to embeddings that disregard them as well. A recent improvement to autoencoders that helps alleviate this problem is the use of perceptual loss. This work investigates perceptual loss from the perspective of encoder embeddings themselves. Autoencoders are trained to embed images from three different computer vision datasets using perceptual loss based on a pretrained model as well as pixel-wise loss. A host of different predictors are trained to perform object positioning and classification on the datasets given the embedded images as input. The two kinds of losses are evaluated by comparing how the predictors performed with embeddings from the differently trained autoencoders. The results show that, in the image domain, the embeddings generated by autoencoders trained with perceptual loss enable more accurate predictions than those trained with element-wise loss. Furthermore, the results show that, on the task of object positioning of a small-scale feature, perceptual loss can improve the results by a factor 10. The experimental setup is available online: https://github.com/guspih/Perceptual-Autoencoders