# NLUT: Neural-based 3D Lookup Tables for Video Photorealistic Style Transfer

arXiv: https://arxiv.org/abs/2303.09170 · HTML: https://mlsanity.com/paper/2303.09170/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 35 references that are in the map.
- First posted on arXiv 16 Mar 2023.

## Builds on (most cited first, 27 of 27)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization](https://mlsanity.com/paper/1703.06868/) (2017)
- [Network In Network](https://mlsanity.com/paper/1312.4400/) (2013)
- [A Neural Algorithm of Artistic Style](https://mlsanity.com/paper/1508.06576/) (2015)
- [Learning Blind Video Temporal Consistency](https://mlsanity.com/paper/1808.00449/) (2018)
- [Deep Photo Style Transfer](https://mlsanity.com/paper/1703.07511/) (2017)
- [Style Transfer by Relaxed Optimal Transport and Self-Similarity](https://mlsanity.com/paper/1904.12785/) (2019)
- [Photorealistic Style Transfer via Wavelet Transforms](https://mlsanity.com/paper/1903.09760/) (2019)
- [Learning Image-adaptive 3D Lookup Tables for High Performance Photo Enhancement in Real-time](https://mlsanity.com/paper/2009.14468/) (2020)
- [StylizedNeRF: Consistent 3D Scene Stylization as Stylized NeRF via 2D-3D Mutual Learning](https://mlsanity.com/paper/2205.12183/) (2022)
- [Robust High-Resolution Video Matting with Temporal Guidance](https://mlsanity.com/paper/2108.11515/) (2021)
- [A Closed-form Solution to Photorealistic Image Stylization](https://mlsanity.com/paper/1802.06474/) (2018)
- [Avatar-Net: Multi-scale Zero-shot Style Transfer by Feature Decoration](https://mlsanity.com/paper/1805.03857/) (2018)
- [Stylizing 3D Scene via Implicit Representation and HyperNetwork](https://mlsanity.com/paper/2105.13016/) (2021)
- [Coherent Online Video Style Transfer](https://mlsanity.com/paper/1703.09211/) (2017)
- [Arbitrary Video Style Transfer via Multi-Channel Correlation](https://mlsanity.com/paper/2009.08003/) (2020)
- [AdaInt: Learning Adaptive Intervals for 3D Lookup Tables on Real-time Image Enhancement](https://mlsanity.com/paper/2204.13983/) (2022)
- [SepLUT: Separable Image-adaptive Lookup Tables for Real-time Image Enhancement](https://mlsanity.com/paper/2207.08351/) (2022)
- [UPST-NeRF: Universal Photorealistic Style Transfer of Neural Radiance Fields for 3D Scene](https://mlsanity.com/paper/2208.07059/) (2022)
- [4D LUT: Learnable Context-Aware 4D Lookup Table for Image Enhancement](https://mlsanity.com/paper/2209.01749/) (2022)
- [Joint Bilateral Learning for Real-time Universal Photorealistic Style Transfer](https://mlsanity.com/paper/2004.10955/) (2020)
- [SinIR: Efficient General Image Manipulation with Single Image Reconstruction](https://mlsanity.com/paper/2106.07140/) (2021)
- [PCA-Based Knowledge Distillation Towards Lightweight and Content-Style Balanced Photorealistic Style Transfer Models](https://mlsanity.com/paper/2203.13452/) (2022)
- [Deep Translation Prior: Test-time Training for Photorealistic Style Transfer](https://mlsanity.com/paper/2112.06150/) (2021)

## Built on by (most cited first, 10 of 10)

- [Video Color Grading via Look-Up Table Generation](https://mlsanity.com/paper/2508.00548/) (2025)
- [MVOC: a training-free multiple video object composition method with diffusion models](https://mlsanity.com/paper/2406.15829/) (2024)
- [SA-LUT: Spatial Adaptive 4D Look-Up Table for Photorealistic Style Transfer](https://mlsanity.com/paper/2506.13465/) (2025)
- [ESA: Energy-Based Shot Assembly Optimization for Automatic Video Editing](https://mlsanity.com/paper/2511.02505/) (2025)
- [NCST: Neural-based Color Style Transfer for Video Retouching](https://mlsanity.com/paper/2411.00335/) (2024)
- [CanonCGT: Reference-Based Color Grading via Canonical Pivot Representation](https://mlsanity.com/paper/2606.01638/) (2026)
- [Multimodal 3D LUT Generation via StatLUT with Statistical Features for Photorealistic Style Transfer](https://mlsanity.com/paper/2607.08227/) (2026)
- [Redistributing the Precision and Content in 3D-LUT-based Inverse Tone-mapping for HDR/WCG Display](https://mlsanity.com/paper/2309.17160/) (2023)
- [GLUT: 3D Gaussian Lookup Table for Continuous Color Transformation](https://mlsanity.com/paper/2605.19889/) (2026)
- [ColorFM: An Optimization-to-Learning Framework for Color Transfer via Flow Matching](https://mlsanity.com/paper/2607.07119/) (2026)

## Abstract (arXiv)

Video photorealistic style transfer is desired to generate videos with a similar photorealistic style to the style image while maintaining temporal consistency. However, existing methods obtain stylized video sequences by performing frame-by-frame photorealistic style transfer, which is inefficient and does not ensure the temporal consistency of the stylized video. To address this issue, we use neural network-based 3D Lookup Tables (LUTs) for the photorealistic transfer of videos, achieving a balance between efficiency and effectiveness. We first train a neural network for generating photorealistic stylized 3D LUTs on a large-scale dataset; then, when performing photorealistic style transfer for a specific video, we select a keyframe and style image in the video as the data source and fine-turn the neural network; finally, we query the 3D LUTs generated by the fine-tuned neural network for the colors in the video, resulting in a super-fast photorealistic style transfer, even processing 8K video takes less than 2 millisecond per frame. The experimental results show that our method not only realizes the photorealistic style transfer of arbitrary style images but also outperforms the existing methods in terms of visual quality and consistency. Project page:https://semchan.github.io/NLUT_Project.