Deep Analog: Open-Set Film Emulation with Reference-Conditioned 3D LUTs
On the map: Mixture-of-experts → Vision mixture-of-experts
Abstract (arXiv)
Film emulation reproduces the look of an analog film stock on a new digital photograph. We target its open-set form -- matching any reference film frame from a single example -- with a 3D lookup table (LUT) predicted from that reference. Real-time image enhancement predicts per-image weights over a fixed bank of 3D LUTs and blends them. We show this is a gated mixture of experts and inherits its failure: trained end-to-end against reconstruction, the gate collapses onto a single expert, so a bank of K LUTs delivers the capacity of one. An entropy term, the enhancement-setting analogue of mixture-of-experts load balancing, restores utilization and recovers about 1 dB PSNR. The deeper constraint survives: a fixed LUT basis is closed-set, freezing the achievable looks at training time. We therefore discard the basis and predict a single 3D LUT as a residual from a reference image (StyleLUTNet), trained by self-supervision on procedurally generated color transforms. The conditional design removes the gate and generalizes open-set to unseen film stocks without paired data or retraining. Around this color backbone we build Deep Analog, a film-emulation pipeline that adds histogram-based tone matching and a physics-informed optical renderer -- multi-scale grain and per-channel halation driven by parameters an inverse network regresses from the reference. On 350 self-supervised pairs the color stage reaches 22.05 dB PSNR / 0.925 SSIM and the full pipeline 21.72 dB / 0.923; the color path runs in 5.2 ms at 1080p (192 FPS) and exports a portable .cube LUT for standard editing tools. A second degeneracy in conditional LUT training -- residual-scale collapse -- shares the root cause and yields a general principle: auxiliary regularization must stay subordinate to reconstruction.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Vision mixture-of-experts, one of 12 directions in Mixture-of-experts. Builds on 20 of its 35 references that are in the map. First posted on arXiv 10 Aug 2026.
Builds on
20 of 35 references.
References · 20
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
- 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
- 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
- 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
- 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
- 2016Perceptual Losses for Real-Time Style Transfer and Super-Resolutioncited by 2,678
- 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,591
- 2017Arbitrary Style Transfer in Real-time with Adaptive Instance Normalizationcited by 1,998
- 2020Pre-Trained Image Processing Transformercited by 891
- 2021Uformer: A General U-Shaped Transformer for Image Restorationcited by 864
- 2017Deep Bilateral Learning for Real-Time Image Enhancementcited by 200
8 more
- 2020Learning Image-adaptive 3D Lookup Tables for High Performance Photo Enhancement in Real-timecited by 163
- 2022AdaInt: Learning Adaptive Intervals for 3D Lookup Tables on Real-time Image Enhancementcited by 70
- 2020How to Train Neural Networks for Flare Removalcited by 50
- 2022SepLUT: Separable Image-adaptive Lookup Tables for Real-time Image Enhancementcited by 50
- 2023Neural Preset for Color Style Transfercited by 36
- 2021StarEnhancer: Learning Real-Time and Style-Aware Image Enhancementcited by 32
- 2023NILUT: Conditional Neural Implicit 3D Lookup Tables for Image Enhancementcited by 31
- 2022Deep-based Film Grain Removal and Synthesiscited by 5
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026A training-free framework for high-fidelity appearance transfer via diffusion transformers
- 2026Realiz3D: 3D Generation Made Photorealistic via Domain-Aware Learning
- 2025FlowLUT: Efficient Image Enhancement via Differentiable LUTs and Iterative Flow Matching
- 2024AC3D: Analyzing and Improving 3D Camera Control in Video Diffusion Transformers
- 2023Zero-1-to-3: Zero-shot One Image to 3D Object
- 2023DiffPortrait3D: Controllable Diffusion for Zero-Shot Portrait View Synthesis
- 2026Stabilizing Camera-Controlled Novel View Synthesis at Inference Time
- 2026RefCompose: Multi-Reference Image Generation via LoRA-Conditioned Diffusion