# Deep Bilateral Learning for Real-Time Image Enhancement

arXiv: https://arxiv.org/abs/1707.02880 · HTML: https://mlsanity.com/paper/1707.02880/

## Key facts

- Cited by 200 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Jul 2017.

## Built on by (most cited first, 30 of 200)

- [NeRF in the Dark: High Dynamic Range View Synthesis from Noisy Raw Images](https://mlsanity.com/paper/2111.13679/) (2021)
- [Learning Enriched Features for Fast Image Restoration and Enhancement](https://mlsanity.com/paper/2205.01649/) (2022)
- [Low-Light Image and Video Enhancement Using Deep Learning: A Survey](https://mlsanity.com/paper/2104.10729/) (2021)
- [InstructIR: High-Quality Image Restoration Following Human Instructions](https://mlsanity.com/paper/2401.16468/) (2024)
- [FeatUp: A Model-Agnostic Framework for Features at Any Resolution](https://mlsanity.com/paper/2403.10516/) (2024)
- [AdaInt: Learning Adaptive Intervals for 3D Lookup Tables on Real-time Image Enhancement](https://mlsanity.com/paper/2204.13983/) (2022)
- [Real-time Image Enhancer via Learnable Spatial-aware 3D Lookup Tables](https://mlsanity.com/paper/2108.08697/) (2021)
- [FourLLIE: Boosting Low-Light Image Enhancement by Fourier Frequency Information](https://mlsanity.com/paper/2308.03033/) (2023)
- [High-Resolution Photorealistic Image Translation in Real-Time: A Laplacian Pyramid Translation Network](https://mlsanity.com/paper/2105.09188/) (2021)
- [PPR10K: A Large-Scale Portrait Photo Retouching Dataset with Human-Region Mask and Group-Level Consistency](https://mlsanity.com/paper/2105.09180/) (2021)
- [High-Resolution Image Harmonization via Collaborative Dual Transformations](https://mlsanity.com/paper/2109.06671/) (2021)
- [Adaptive Unfolding Total Variation Network for Low-Light Image Enhancement](https://mlsanity.com/paper/2110.00984/) (2021)
- [SepLUT: Separable Image-adaptive Lookup Tables for Real-time Image Enhancement](https://mlsanity.com/paper/2207.08351/) (2022)
- [TransFill: Reference-guided Image Inpainting by Merging Multiple Color and Spatial Transformations](https://mlsanity.com/paper/2103.15982/) (2021)
- [Bilateral Grid Learning for Stereo Matching Networks](https://mlsanity.com/paper/2101.01601/) (2021)
- [DCCF: Deep Comprehensible Color Filter Learning Framework for High-Resolution Image Harmonization](https://mlsanity.com/paper/2207.04788/) (2022)
- [Deep Learning for HDR Imaging: State-of-the-Art and Future Trends](https://mlsanity.com/paper/2110.10394/) (2021)
- [Learning RAW-to-sRGB Mappings with Inaccurately Aligned Supervision](https://mlsanity.com/paper/2108.08119/) (2021)
- [Lighting up NeRF via Unsupervised Decomposition and Enhancement](https://mlsanity.com/paper/2307.10664/) (2023)
- [Neural Preset for Color Style Transfer](https://mlsanity.com/paper/2303.13511/) (2023)
- [4D LUT: Learnable Context-Aware 4D Lookup Table for Image Enhancement](https://mlsanity.com/paper/2209.01749/) (2022)
- [StarEnhancer: Learning Real-Time and Style-Aware Image Enhancement](https://mlsanity.com/paper/2107.12898/) (2021)
- [NILUT: Conditional Neural Implicit 3D Lookup Tables for Image Enhancement](https://mlsanity.com/paper/2306.11920/) (2023)
- [A New Journey from SDRTV to HDRTV](https://mlsanity.com/paper/2108.07978/) (2021)
- [Bilateral Guided Radiance Field Processing](https://mlsanity.com/paper/2406.00448/) (2024)
- [Blind Video Deflickering by Neural Filtering with a Flawed Atlas](https://mlsanity.com/paper/2303.08120/) (2023)
- [ReLLIE: Deep Reinforcement Learning for Customized Low-Light Image Enhancement](https://mlsanity.com/paper/2107.05830/) (2021)
- [Towards Low Light Enhancement with RAW Images](https://mlsanity.com/paper/2112.14022/) (2021)
- [Semi-supervised Parametric Real-world Image Harmonization](https://mlsanity.com/paper/2303.00157/) (2023)
- [Spatial-Separated Curve Rendering Network for Efficient and High-Resolution Image Harmonization](https://mlsanity.com/paper/2109.05750/) (2021)

## Abstract (arXiv)

Performance is a critical challenge in mobile image processing. Given a reference imaging pipeline, or even human-adjusted pairs of images, we seek to reproduce the enhancements and enable real-time evaluation. For this, we introduce a new neural network architecture inspired by bilateral grid processing and local affine color transforms. Using pairs of input/output images, we train a convolutional neural network to predict the coefficients of a locally-affine model in bilateral space. Our architecture learns to make local, global, and content-dependent decisions to approximate the desired image transformation. At runtime, the neural network consumes a low-resolution version of the input image, produces a set of affine transformations in bilateral space, upsamples those transformations in an edge-preserving fashion using a new slicing node, and then applies those upsampled transformations to the full-resolution image. Our algorithm processes high-resolution images on a smartphone in milliseconds, provides a real-time viewfinder at 1080p resolution, and matches the quality of state-of-the-art approximation techniques on a large class of image operators. Unlike previous work, our model is trained off-line from data and therefore does not require access to the original operator at runtime. This allows our model to learn complex, scene-dependent transformations for which no reference implementation is available, such as the photographic edits of a human retoucher.