# The Contextual Loss for Image Transformation with Non-Aligned Data

arXiv: https://arxiv.org/abs/1803.02077 · HTML: https://mlsanity.com/paper/1803.02077/

## Key facts

- Cited by 135 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Mar 2018.

## Built on by (most cited first, 30 of 135)

- [Make-It-3D: High-Fidelity 3D Creation from A Single Image with Diffusion Prior](https://mlsanity.com/paper/2303.14184/) (2023)
- [Deep ViT Features as Dense Visual Descriptors](https://mlsanity.com/paper/2112.05814/) (2021)
- [Deblurring via Stochastic Refinement](https://mlsanity.com/paper/2112.02475/) (2021)
- [Splicing ViT Features for Semantic Appearance Transfer](https://mlsanity.com/paper/2201.00424/) (2022)
- [Point-Based Neural Rendering with Per-View Optimization](https://mlsanity.com/paper/2109.02369/) (2021)
- [Inversion by Direct Iteration: An Alternative to Denoising Diffusion for Image Restoration](https://mlsanity.com/paper/2303.11435/) (2023)
- [DreamCraft3D: Hierarchical 3D Generation with Bootstrapped Diffusion Prior](https://mlsanity.com/paper/2310.16818/) (2023)
- [ARF: Artistic Radiance Fields](https://mlsanity.com/paper/2206.06360/) (2022)
- [MASA-SR: Matching Acceleration and Spatial Adaptation for Reference-Based Image Super-Resolution](https://mlsanity.com/paper/2106.02299/) (2021)
- [Multiscale Structure Guided Diffusion for Image Deblurring](https://mlsanity.com/paper/2212.01789/) (2022)
- [The Spatially-Correlative Loss for Various Image Translation Tasks](https://mlsanity.com/paper/2104.00854/) (2021)
- [Pastiche Master: Exemplar-Based High-Resolution Portrait Style Transfer](https://mlsanity.com/paper/2203.13248/) (2022)
- [Real-World Single Image Super-Resolution: A Brief Review](https://mlsanity.com/paper/2103.02368/) (2021)
- [StyleMesh: Style Transfer for Indoor 3D Scene Reconstructions](https://mlsanity.com/paper/2112.01530/) (2021)
- [Semantic Image Matting](https://mlsanity.com/paper/2104.08201/) (2021)
- [Towards Vivid and Diverse Image Colorization with Generative Color Prior](https://mlsanity.com/paper/2108.08826/) (2021)
- [Rethinking and Improving the Robustness of Image Style Transfer](https://mlsanity.com/paper/2104.05623/) (2021)
- [Cross Attention Based Style Distribution for Controllable Person Image Synthesis](https://mlsanity.com/paper/2208.00712/) (2022)
- [Unbalanced Feature Transport for Exemplar-based Image Translation](https://mlsanity.com/paper/2106.10482/) (2021)
- [Domain-Aware Universal Style Transfer](https://mlsanity.com/paper/2108.04441/) (2021)
- [QS-Attn: Query-Selected Attention for Contrastive Learning in I2I Translation](https://mlsanity.com/paper/2203.08483/) (2022)
- [3D GAN Inversion with Facial Symmetry Prior](https://mlsanity.com/paper/2211.16927/) (2022)
- [Cycle3D: High-quality and Consistent Image-to-3D Generation via Generation-Reconstruction Cycle](https://mlsanity.com/paper/2407.19548/) (2024)
- [Learning Degradation Representations for Image Deblurring](https://mlsanity.com/paper/2208.05244/) (2022)
- [Dual-Camera Super-Resolution with Aligned Attention Modules](https://mlsanity.com/paper/2109.01349/) (2021)
- [AesPA-Net: Aesthetic Pattern-Aware Style Transfer Networks](https://mlsanity.com/paper/2307.09724/) (2023)
- [HiFi-123: Towards High-fidelity One Image to 3D Content Generation](https://mlsanity.com/paper/2310.06744/) (2023)
- [Reference-guided Pseudo-Label Generation for Medical Semantic Segmentation](https://mlsanity.com/paper/2112.00735/) (2021)
- [DiffFashion: Reference-based Fashion Design with Structure-aware Transfer by Diffusion Models](https://mlsanity.com/paper/2302.06826/) (2023)
- [Attention Distillation: A Unified Approach to Visual Characteristics Transfer](https://mlsanity.com/paper/2502.20235/) (2025)

## Abstract (arXiv)

Feed-forward CNNs trained for image transformation problems rely on loss functions that measure the similarity between the generated image and a target image. Most of the common loss functions assume that these images are spatially aligned and compare pixels at corresponding locations. However, for many tasks, aligned training pairs of images will not be available. We present an alternative loss function that does not require alignment, thus providing an effective and simple solution for a new space of problems. Our loss is based on both context and semantics -- it compares regions with similar semantic meaning, while considering the context of the entire image. Hence, for example, when transferring the style of one face to another, it will translate eyes-to-eyes and mouth-to-mouth. Our code can be found at https://www.github.com/roimehrez/contextualLoss