# High-Resolution Photorealistic Image Translation in Real-Time: A Laplacian Pyramid Translation Network

arXiv: https://arxiv.org/abs/2105.09188 · HTML: https://mlsanity.com/paper/2105.09188/

## Key facts

- Cited by 61 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 33 references that are in the map.
- First posted on arXiv 19 May 2021.

## Builds on (most cited first, 25 of 25)

- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization](https://mlsanity.com/paper/1703.06868/) (2017)
- [Unsupervised Domain Adaptation by Backpropagation](https://mlsanity.com/paper/1409.7495/) (2014)
- [High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs](https://mlsanity.com/paper/1711.11585/) (2017)
- [CyCADA: Cycle-Consistent Adversarial Domain Adaptation](https://mlsanity.com/paper/1711.03213/) (2017)
- [Least Squares Generative Adversarial Networks](https://mlsanity.com/paper/1611.04076/) (2016)
- [Multimodal Unsupervised Image-to-Image Translation](https://mlsanity.com/paper/1804.04732/) (2018)
- [Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks](https://mlsanity.com/paper/1703.10593/) (2017)
- [Unsupervised Image-to-Image Translation Networks](https://mlsanity.com/paper/1703.00848/) (2017)
- [Deep Laplacian Pyramid Networks for Fast and Accurate Super-Resolution](https://mlsanity.com/paper/1704.03915/) (2017)
- [Toward Multimodal Image-to-Image Translation](https://mlsanity.com/paper/1711.11586/) (2017)
- [Diverse Image-to-Image Translation via Disentangled Representations](https://mlsanity.com/paper/1808.00948/) (2018)
- [Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks](https://mlsanity.com/paper/1506.05751/) (2015)
- [Deep Bilateral Learning for Real-Time Image Enhancement](https://mlsanity.com/paper/1707.02880/) (2017)
- [Deep Photo Style Transfer](https://mlsanity.com/paper/1703.07511/) (2017)
- [Photorealistic Style Transfer via Wavelet Transforms](https://mlsanity.com/paper/1903.09760/) (2019)
- [A Closed-form Solution to Photorealistic Image Stylization](https://mlsanity.com/paper/1802.06474/) (2018)
- [Real-Time User-Guided Image Colorization with Learned Deep Priors](https://mlsanity.com/paper/1705.02999/) (2017)
- [Coherent Online Video Style Transfer](https://mlsanity.com/paper/1703.09211/) (2017)
- [Learning Affinity via Spatial Propagation Networks](https://mlsanity.com/paper/1710.01020/) (2017)
- [A Unified Feature Disentangler for Multi-Domain Image Translation and Manipulation](https://mlsanity.com/paper/1809.01361/) (2018)
- [Image-to-image translation for cross-domain disentanglement](https://mlsanity.com/paper/1805.09730/) (2018)
- [Exemplar Guided Unsupervised Image-to-Image Translation with Semantic Consistency](https://mlsanity.com/paper/1805.11145/) (2018)
- [Harmonic Unpaired Image-to-image Translation](https://mlsanity.com/paper/1902.09727/) (2019)

## Built on by (most cited first, 30 of 61)

- [Thin-Plate Spline Motion Model for Image Animation](https://mlsanity.com/paper/2203.14367/) (2022)
- [DEA-Net: Single image dehazing based on detail-enhanced convolution and content-guided attention](https://mlsanity.com/paper/2301.04805/) (2023)
- [Wave-Mamba: Wavelet State Space Model for Ultra-High-Definition Low-Light Image Enhancement](https://mlsanity.com/paper/2408.01276/) (2024)
- [High-Resolution Document Shadow Removal via A Large-Scale Real-World Dataset and A Frequency-Aware Shadow Erasing Net](https://mlsanity.com/paper/2308.14221/) (2023)
- [TGFuse: An Infrared and Visible Image Fusion Approach Based on Transformer and Generative Adversarial Network](https://mlsanity.com/paper/2201.10147/) (2022)
- [MEGANet: Multi-Scale Edge-Guided Attention Network for Weak Boundary Polyp Segmentation](https://mlsanity.com/paper/2309.03329/) (2023)
- [A Large-scale Film Style Dataset for Learning Multi-frequency Driven Film Enhancement](https://mlsanity.com/paper/2301.08880/) (2023)
- [Degradation-invariant Enhancement of Fundus Images via Pyramid Constraint Network](https://mlsanity.com/paper/2210.09606/) (2022)
- [Devignet: High-Resolution Vignetting Removal via a Dual Aggregated Fusion Transformer With Adaptive Channel Expansion](https://mlsanity.com/paper/2308.13739/) (2023)
- [MFDNet: Multi-Frequency Deflare Network for Efficient Nighttime Flare Removal](https://mlsanity.com/paper/2406.18079/) (2024)
- [Content-Style Decoupling for Unsupervised Makeup Transfer without Generating Pseudo Ground Truth](https://mlsanity.com/paper/2405.17240/) (2024)
- [Single UHD Image Dehazing via Interpretable Pyramid Network](https://mlsanity.com/paper/2202.08589/) (2022)
- [MedPrompt: Cross-Modal Prompting for Multi-Task Medical Image Translation](https://mlsanity.com/paper/2310.02663/) (2023)
- [An I2I Inpainting Approach for Efficient Channel Knowledge Map Construction](https://mlsanity.com/paper/2406.09822/) (2024)
- [UWFormer: Underwater Image Enhancement via a Semi-Supervised Multi-Scale Transformer](https://mlsanity.com/paper/2310.20210/) (2023)
- [4K-Resolution Photo Exposure Correction at 125 FPS with ~8K Parameters](https://mlsanity.com/paper/2311.08759/) (2023)
- [Neural Color Operators for Sequential Image Retouching](https://mlsanity.com/paper/2207.08080/) (2022)
- [MixNet: Efficient Global Modeling for Ultra-High-Definition Image Restoration](https://mlsanity.com/paper/2401.10666/) (2024)
- [MM2Latent: Text-to-facial image generation and editing in GANs with multimodal assistance](https://mlsanity.com/paper/2409.11010/) (2024)
- [Fast and High-Quality Image Denoising via Malleable Convolutions](https://mlsanity.com/paper/2201.00392/) (2022)
- [Every Pixel Has its Moments: Ultra-High-Resolution Unpaired Image-to-Image Translation via Dense Normalization](https://mlsanity.com/paper/2407.04245/) (2024)
- [PixelFolder: An Efficient Progressive Pixel Synthesis Network for Image Generation](https://mlsanity.com/paper/2204.00833/) (2022)
- [LP-IOANet: Efficient High Resolution Document Shadow Removal](https://mlsanity.com/paper/2303.12862/) (2023)
- [Freditor: High-Fidelity and Transferable NeRF Editing by Frequency Decomposition](https://mlsanity.com/paper/2404.02514/) (2024)
- [Point-and-Shoot All-in-Focus Photo Synthesis from Smartphone Camera Pair](https://mlsanity.com/paper/2304.04917/) (2023)
- [Learning Adaptive Lighting via Channel-Aware Guidance](https://mlsanity.com/paper/2412.01493/) (2024)
- [Multi-Curve Translator for High-Resolution Photorealistic Image Translation](https://mlsanity.com/paper/2203.07756/) (2022)
- [Adaptive Mask-based Pyramid Network for Realistic Bokeh Rendering](https://mlsanity.com/paper/2210.16078/) (2022)
- [CLIP Guided Image-perceptive Prompt Learning for Image Enhancement](https://mlsanity.com/paper/2311.03943/) (2023)
- [S2ST: Image-to-Image Translation in the Seed Space of Latent Diffusion](https://mlsanity.com/paper/2312.00116/) (2023)

## Abstract (arXiv)

Existing image-to-image translation (I2IT) methods are either constrained to low-resolution images or long inference time due to their heavy computational burden on the convolution of high-resolution feature maps. In this paper, we focus on speeding-up the high-resolution photorealistic I2IT tasks based on closed-form Laplacian pyramid decomposition and reconstruction. Specifically, we reveal that the attribute transformations, such as illumination and color manipulation, relate more to the low-frequency component, while the content details can be adaptively refined on high-frequency components. We consequently propose a Laplacian Pyramid Translation Network (LPTN) to simultaneously perform these two tasks, where we design a lightweight network for translating the low-frequency component with reduced resolution and a progressive masking strategy to efficiently refine the high-frequency ones. Our model avoids most of the heavy computation consumed by processing high-resolution feature maps and faithfully preserves the image details. Extensive experimental results on various tasks demonstrate that the proposed method can translate 4K images in real-time using one normal GPU while achieving comparable transformation performance against existing methods. Datasets and codes are available: https://github.com/csjliang/LPTN.