# Deep Translation Prior: Test-time Training for Photorealistic Style Transfer

arXiv: https://arxiv.org/abs/2112.06150 · HTML: https://mlsanity.com/paper/2112.06150/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 41 of its 53 references that are in the map.
- First posted on arXiv 12 Dec 2021.

## Builds on (most cited first, 30 of 41)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization](https://mlsanity.com/paper/1703.06868/) (2017)
- [Understanding deep learning requires rethinking generalization](https://mlsanity.com/paper/1611.03530/) (2016)
- [Semantic Image Synthesis with Spatially-Adaptive Normalization](https://mlsanity.com/paper/1903.07291/) (2019)
- [Deep Image Prior](https://mlsanity.com/paper/1711.10925/) (2017)
- [Multimodal Unsupervised Image-to-Image Translation](https://mlsanity.com/paper/1804.04732/) (2018)
- [Contrastive Learning for Unpaired Image-to-Image Translation](https://mlsanity.com/paper/2007.15651/) (2020)
- [Unsupervised Image-to-Image Translation Networks](https://mlsanity.com/paper/1703.00848/) (2017)
- [PULSE: Self-Supervised Photo Upsampling via Latent Space Exploration of Generative Models](https://mlsanity.com/paper/2003.03808/) (2020)
- [Universal Style Transfer via Feature Transforms](https://mlsanity.com/paper/1705.08086/) (2017)
- [SinGAN: Learning a Generative Model from a Single Natural Image](https://mlsanity.com/paper/1905.01164/) (2019)
- [On the "steerability" of generative adversarial networks](https://mlsanity.com/paper/1907.07171/) (2019)
- [Photographic Image Synthesis with Cascaded Refinement Networks](https://mlsanity.com/paper/1707.09405/) (2017)
- [Improved Texture Networks: Maximizing Quality and Diversity in Feed-forward Stylization and Texture Synthesis](https://mlsanity.com/paper/1701.02096/) (2017)
- [Swapping Autoencoder for Deep Image Manipulation](https://mlsanity.com/paper/2007.00653/) (2020)
- [Arbitrary Style Transfer with Style-Attentional Networks](https://mlsanity.com/paper/1812.02342/) (2018)
- [Image Processing Using Multi-Code GAN Prior](https://mlsanity.com/paper/1912.07116/) (2019)
- [AdaAttN: Revisit Attention Mechanism in Arbitrary Neural Style Transfer](https://mlsanity.com/paper/2108.03647/) (2021)
- [Deep Photo Style Transfer](https://mlsanity.com/paper/1703.07511/) (2017)
- [Style Transfer by Relaxed Optimal Transport and Self-Similarity](https://mlsanity.com/paper/1904.12785/) (2019)
- [Photorealistic Style Transfer via Wavelet Transforms](https://mlsanity.com/paper/1903.09760/) (2019)
- [PieAPP: Perceptual Image-Error Assessment through Pairwise Preference](https://mlsanity.com/paper/1806.02067/) (2018)
- [Convolutional neural network architecture for geometric matching](https://mlsanity.com/paper/1703.05593/) (2017)
- [The Contextual Loss for Image Transformation with Non-Aligned Data](https://mlsanity.com/paper/1803.02077/) (2018)
- [A Closed-form Solution to Photorealistic Image Stylization](https://mlsanity.com/paper/1802.06474/) (2018)

## Built on by (most cited first, 8 of 8)

- [ArtFID: Quantitative Evaluation of Neural Style Transfer](https://mlsanity.com/paper/2207.12280/) (2022)
- [StylizedGS: Controllable Stylization for 3D Gaussian Splatting](https://mlsanity.com/paper/2404.05220/) (2024)
- [MIDMs: Matching Interleaved Diffusion Models for Exemplar-based Image Translation](https://mlsanity.com/paper/2209.11047/) (2022)
- [Introducing Intermediate Domains for Effective Self-Training during Test-Time](https://mlsanity.com/paper/2208.07736/) (2022)
- [NLUT: Neural-based 3D Lookup Tables for Video Photorealistic Style Transfer](https://mlsanity.com/paper/2303.09170/) (2023)
- [Controllable Style Transfer via Test-time Training of Implicit Neural Representation](https://mlsanity.com/paper/2210.07762/) (2022)
- [Unsupervised Scene Sketch to Photo Synthesis](https://mlsanity.com/paper/2209.02834/) (2022)
- [Line Search-Based Feature Transformation for Fast, Stable, and Tunable Content-Style Control in Photorealistic Style Transfer](https://mlsanity.com/paper/2210.05996/) (2022)

## Abstract (arXiv)

Recent techniques to solve photorealistic style transfer within deep convolutional neural networks (CNNs) generally require intensive training from large-scale datasets, thus having limited applicability and poor generalization ability to unseen images or styles. To overcome this, we propose a novel framework, dubbed Deep Translation Prior (DTP), to accomplish photorealistic style transfer through test-time training on given input image pair with untrained networks, which learns an image pair-specific translation prior and thus yields better performance and generalization. Tailored for such test-time training for style transfer, we present novel network architectures, with two sub-modules of correspondence and generation modules, and loss functions consisting of contrastive content, style, and cycle consistency losses. Our framework does not require offline training phase for style transfer, which has been one of the main challenges in existing methods, but the networks are to be solely learned during test-time. Experimental results prove that our framework has a better generalization ability to unseen image pairs and even outperforms the state-of-the-art methods.