# Maintaining Natural Image Statistics with the Contextual Loss

arXiv: https://arxiv.org/abs/1803.04626 · HTML: https://mlsanity.com/paper/1803.04626/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Mar 2018.

## Built on by (most cited first, 19 of 19)

- [Deblurring via Stochastic Refinement](https://mlsanity.com/paper/2112.02475/) (2021)
- [Inversion by Direct Iteration: An Alternative to Denoising Diffusion for Image Restoration](https://mlsanity.com/paper/2303.11435/) (2023)
- [MASA-SR: Matching Acceleration and Spatial Adaptation for Reference-Based Image Super-Resolution](https://mlsanity.com/paper/2106.02299/) (2021)
- [Multiscale Structure Guided Diffusion for Image Deblurring](https://mlsanity.com/paper/2212.01789/) (2022)
- [Token Pooling in Vision Transformers](https://mlsanity.com/paper/2110.03860/) (2021)
- [Dual-Camera Super-Resolution with Aligned Attention Modules](https://mlsanity.com/paper/2109.01349/) (2021)
- [Best-Buddy GANs for Highly Detailed Image Super-Resolution](https://mlsanity.com/paper/2103.15295/) (2021)
- [Modulated Contrast for Versatile Image Synthesis](https://mlsanity.com/paper/2203.09333/) (2022)
- [LTT-GAN: Looking Through Turbulence by Inverting GANs](https://mlsanity.com/paper/2112.02379/) (2021)
- [Reference-based Video Super-Resolution Using Multi-Camera Video Triplets](https://mlsanity.com/paper/2203.14537/) (2022)
- [Perception-Distortion Balanced ADMM Optimization for Single-Image Super-Resolution](https://mlsanity.com/paper/2208.03324/) (2022)
- [Clean Images are Hard to Reblur: Exploiting the Ill-Posed Inverse Task for Dynamic Scene Deblurring](https://mlsanity.com/paper/2104.12665/) (2021)
- [Training Generative Image Super-Resolution Models by Wavelet-Domain Losses Enables Better Control of Artifacts](https://mlsanity.com/paper/2402.19215/) (2024)
- [Deep Neural Network-based Enhancement for Image and Video Streaming Systems: A Survey and Future Directions](https://mlsanity.com/paper/2106.03727/) (2021)
- [Hallucination Score: Towards Mitigating Hallucinations in Generative Image Super-Resolution](https://mlsanity.com/paper/2507.14367/) (2025)
- [Deep Semantic Statistics Matching (D2SM) Denoising Network](https://mlsanity.com/paper/2207.09302/) (2022)
- [MGHF: Multi-Granular High-Frequency Perceptual Loss for Image Super-Resolution](https://mlsanity.com/paper/2411.13548/) (2024)
- [Real-World Video for Zoom Enhancement based on Spatio-Temporal Coupling](https://mlsanity.com/paper/2306.13875/) (2023)
- [StableSemantics: A Synthetic Language-Vision Dataset of Semantic Representations in Naturalistic Images](https://mlsanity.com/paper/2406.13735/) (2024)

## Abstract (arXiv)

Maintaining natural image statistics is a crucial factor in restoration and generation of realistic looking images. When training CNNs, photorealism is usually attempted by adversarial training (GAN), that pushes the output images to lie on the manifold of natural images. GANs are very powerful, but not perfect. They are hard to train and the results still often suffer from artifacts. In this paper we propose a complementary approach, that could be applied with or without GAN, whose goal is to train a feed-forward CNN to maintain natural internal statistics. We look explicitly at the distribution of features in an image and train the network to generate images with natural feature distributions. Our approach reduces by orders of magnitude the number of images required for training and achieves state-of-the-art results on both single-image super-resolution, and high-resolution surface normal estimation.