# Dynamic Texture Transfer using PatchMatch and Transformers

arXiv: https://arxiv.org/abs/2402.00606 · HTML: https://mlsanity.com/paper/2402.00606/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 22 references that are in the map.
- First posted on arXiv 1 Feb 2024.

## Builds on (most cited first, 12 of 12)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [Taming Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2012.09841/) (2020)
- [First Order Motion Model for Image Animation](https://mlsanity.com/paper/2003.00196/) (2020)
- [VideoGPT: Video Generation using VQ-VAE and Transformers](https://mlsanity.com/paper/2104.10157/) (2021)
- [SinGAN: Learning a Generative Model from a Single Natural Image](https://mlsanity.com/paper/1905.01164/) (2019)
- [Controllable Artistic Text Style Transfer via Shape-Matching GAN](https://mlsanity.com/paper/1905.01354/) (2019)
- [Awesome Typography: Statistics-Based Text Effects Transfer](https://mlsanity.com/paper/1611.09026/) (2016)
- [TET-GAN: Text Effects Transfer via Stylization and Destylization](https://mlsanity.com/paper/1812.06384/) (2018)
- [Two-Stream Convolutional Networks for Dynamic Texture Synthesis](https://mlsanity.com/paper/1706.06982/) (2017)
- [Context-Aware Text-Based Binary Image Stylization and Synthesis](https://mlsanity.com/paper/1810.03767/) (2018)
- [Learning to Transfer Visual Effects from Videos to Images](https://mlsanity.com/paper/2012.01642/) (2020)

## Built on by (most cited first, 3 of 3)

- [Intelligent Artistic Typography: A Comprehensive Review of Artistic Text Design and Generation](https://mlsanity.com/paper/2407.14774/) (2024)
- [VitaGlyph: Vitalizing Artistic Typography with Flexible Dual-branch Diffusion Models](https://mlsanity.com/paper/2410.01738/) (2024)
- [GT2-GS: Geometry-aware Texture Transfer for Gaussian Splatting](https://mlsanity.com/paper/2505.15208/) (2025)

## Abstract (arXiv)

How to automatically transfer the dynamic texture of a given video to the target still image is a challenging and ongoing problem. In this paper, we propose to handle this task via a simple yet effective model that utilizes both PatchMatch and Transformers. The key idea is to decompose the task of dynamic texture transfer into two stages, where the start frame of the target video with the desired dynamic texture is synthesized in the first stage via a distance map guided texture transfer module based on the PatchMatch algorithm. Then, in the second stage, the synthesized image is decomposed into structure-agnostic patches, according to which their corresponding subsequent patches can be predicted by exploiting the powerful capability of Transformers equipped with VQ-VAE for processing long discrete sequences. After getting all those patches, we apply a Gaussian weighted average merging strategy to smoothly assemble them into each frame of the target stylized video. Experimental results demonstrate the effectiveness and superiority of the proposed method in dynamic texture transfer compared to the state of the art.