# Distilling Style from Image Pairs for Global Forward and Inverse Tone Mapping

arXiv: https://arxiv.org/abs/2209.15165 · HTML: https://mlsanity.com/paper/2209.15165/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 51 references that are in the map.
- First posted on arXiv 30 Sep 2022.

## Builds on (most cited first, 24 of 24)

- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains](https://mlsanity.com/paper/2006.10739/) (2020)
- [Variational Inference with Normalizing Flows](https://mlsanity.com/paper/1505.05770/) (2015)
- [Glow: Generative Flow with Invertible 1x1 Convolutions](https://mlsanity.com/paper/1807.03039/) (2018)
- [SRFlow: Learning the Super-Resolution Space with Normalizing Flow](https://mlsanity.com/paper/2006.14200/) (2020)
- [Deep Bilateral Learning for Real-Time Image Enhancement](https://mlsanity.com/paper/1707.02880/) (2017)
- [HDR image reconstruction from a single exposure using deep CNNs](https://mlsanity.com/paper/1710.07480/) (2017)
- [Learning Image-adaptive 3D Lookup Tables for High Performance Photo Enhancement in Real-time](https://mlsanity.com/paper/2009.14468/) (2020)
- [DSLR-Quality Photos on Mobile Devices with Deep Convolutional Networks](https://mlsanity.com/paper/1704.02470/) (2017)
- [Single-Image HDR Reconstruction by Learning to Reverse the Camera Pipeline](https://mlsanity.com/paper/2004.01179/) (2020)
- [DeepLPF: Deep Local Parametric Filters for Image Enhancement](https://mlsanity.com/paper/2003.13985/) (2020)
- [ExpandNet: A Deep Convolutional Neural Network for High Dynamic Range Expansion from Low Dynamic Range Content](https://mlsanity.com/paper/1803.02266/) (2018)
- [Single Image HDR Reconstruction Using a CNN with Masked Features and Perceptual Loss](https://mlsanity.com/paper/2005.07335/) (2020)
- [Conditional Sequential Modulation for Efficient Global Image Retouching](https://mlsanity.com/paper/2009.10390/) (2020)
- [Augmented Normalizing Flows: Bridging the Gap Between Generative Flows and Latent Variable Models](https://mlsanity.com/paper/2002.07101/) (2020)
- [Distort-and-Recover: Color Enhancement using Deep Reinforcement Learning](https://mlsanity.com/paper/1804.04450/) (2018)
- [Deep SR-ITM: Joint Learning of Super-Resolution and Inverse Tone-Mapping for 4K UHD HDR Applications](https://mlsanity.com/paper/1904.11176/) (2019)
- [StarEnhancer: Learning Real-Time and Style-Aware Image Enhancement](https://mlsanity.com/paper/2107.12898/) (2021)
- [A New Journey from SDRTV to HDRTV](https://mlsanity.com/paper/2108.07978/) (2021)
- [Deep Tone Mapping Operator for High Dynamic Range Images](https://mlsanity.com/paper/1908.04197/) (2019)
- [Conditional Invertible Neural Networks for Diverse Image-to-Image Translation](https://mlsanity.com/paper/2105.02104/) (2021)
- [Conditional Invertible Neural Networks for Medical Imaging](https://mlsanity.com/paper/2110.14520/) (2021)
- [How to cheat with metrics in single-image HDR reconstruction](https://mlsanity.com/paper/2108.08713/) (2021)

## Built on by (most cited first, 5 of 5)

- [Learning a Practical SDR-to-HDRTV Up-conversion using New Dataset and Degradation Models](https://mlsanity.com/paper/2303.13031/) (2023)
- [FrePolad: Frequency-Rectified Point Latent Diffusion for Point Cloud Generation](https://mlsanity.com/paper/2311.12090/) (2023)
- [Neural Lens Modeling](https://mlsanity.com/paper/2304.04848/) (2023)
- [One-shot Detail Retouching with Patch Space Neural Transformation Blending](https://mlsanity.com/paper/2210.01217/) (2022)
- [Redistributing the Precision and Content in 3D-LUT-based Inverse Tone-mapping for HDR/WCG Display](https://mlsanity.com/paper/2309.17160/) (2023)

## Abstract (arXiv)

Many image enhancement or editing operations, such as forward and inverse tone mapping or color grading, do not have a unique solution, but instead a range of solutions, each representing a different style. Despite this, existing learning-based methods attempt to learn a unique mapping, disregarding this style. In this work, we show that information about the style can be distilled from collections of image pairs and encoded into a 2- or 3-dimensional vector. This gives us not only an efficient representation but also an interpretable latent space for editing the image style. We represent the global color mapping between a pair of images as a custom normalizing flow, conditioned on a polynomial basis of the pixel color. We show that such a network is more effective than PCA or VAE at encoding image style in low-dimensional space and lets us obtain an accuracy close to 40 dB, which is about 7-10 dB improvement over the state-of-the-art methods.