# Combining Markov Random Fields and Convolutional Neural Networks for Image Synthesis

arXiv: https://arxiv.org/abs/1601.04589 · HTML: https://mlsanity.com/paper/1601.04589/

## Key facts

- Cited by 102 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Jan 2016.

## Built on by (most cited first, 30 of 102)

- [More Control for Free! Image Synthesis with Semantic Diffusion Guidance](https://mlsanity.com/paper/2112.05744/) (2021)
- [Splicing ViT Features for Semantic Appearance Transfer](https://mlsanity.com/paper/2201.00424/) (2022)
- [ARF: Artistic Radiance Fields](https://mlsanity.com/paper/2206.06360/) (2022)
- [ArtFlow: Unbiased Image Style Transfer via Reversible Neural Flows](https://mlsanity.com/paper/2103.16877/) (2021)
- [Pastiche Master: Exemplar-Based High-Resolution Portrait Style Transfer](https://mlsanity.com/paper/2203.13248/) (2022)
- [StyleMesh: Style Transfer for Indoor 3D Scene Reconstructions](https://mlsanity.com/paper/2112.01530/) (2021)
- [Drafting and Revision: Laplacian Pyramid Network for Fast High-Quality Artistic Style Transfer](https://mlsanity.com/paper/2104.05376/) (2021)
- [Neural Preset for Color Style Transfer](https://mlsanity.com/paper/2303.13511/) (2023)
- [Feature-based Style Randomization for Domain Generalization](https://mlsanity.com/paper/2106.03171/) (2021)
- [NeRFFaceEditing: Disentangled Face Editing in Neural Radiance Fields](https://mlsanity.com/paper/2211.07968/) (2022)
- [DiffFashion: Reference-based Fashion Design with Structure-aware Transfer by Diffusion Models](https://mlsanity.com/paper/2302.06826/) (2023)
- [Attention Distillation: A Unified Approach to Visual Characteristics Transfer](https://mlsanity.com/paper/2502.20235/) (2025)
- [Learning to Generate Realistic Noisy Images via Pixel-level Noise-aware Adversarial Training](https://mlsanity.com/paper/2204.02844/) (2022)
- [Learning Graph Neural Networks for Image Style Transfer](https://mlsanity.com/paper/2207.11681/) (2022)
- [Generating natural images with direct Patch Distributions Matching](https://mlsanity.com/paper/2203.11862/) (2022)
- [In the light of feature distributions: moment matching for Neural Style Transfer](https://mlsanity.com/paper/2103.07208/) (2021)
- [RFormer: Transformer-based Generative Adversarial Network for Real Fundus Image Restoration on A New Clinical Benchmark](https://mlsanity.com/paper/2201.00466/) (2022)
- [Semantic Segmentation in Art Paintings](https://mlsanity.com/paper/2203.03238/) (2022)
- [Neural Neighbor Style Transfer](https://mlsanity.com/paper/2203.13215/) (2022)
- [Exemplar-Based 3D Portrait Stylization](https://mlsanity.com/paper/2104.14559/) (2021)
- [AesUST: Towards Aesthetic-Enhanced Universal Style Transfer](https://mlsanity.com/paper/2208.13016/) (2022)
- [SOUP-GAN: Super-Resolution MRI Using Generative Adversarial Networks](https://mlsanity.com/paper/2106.02599/) (2021)
- [Improving Nighttime Driving-Scene Segmentation via Dual Image-adaptive Learnable Filters](https://mlsanity.com/paper/2207.01331/) (2022)
- [Training a Task-Specific Image Reconstruction Loss](https://mlsanity.com/paper/2103.14616/) (2021)
- [All-to-key Attention for Arbitrary Style Transfer](https://mlsanity.com/paper/2212.04105/) (2022)
- [Geometry Transfer for Stylizing Radiance Fields](https://mlsanity.com/paper/2402.00863/) (2024)
- [Spatial Mixture-of-Experts](https://mlsanity.com/paper/2211.13491/) (2022)
- [CoARF: Controllable 3D Artistic Style Transfer for Radiance Fields](https://mlsanity.com/paper/2404.14967/) (2024)
- [Texture Reformer: Towards Fast and Universal Interactive Texture Transfer](https://mlsanity.com/paper/2112.02788/) (2021)
- [Reference-based Controllable Scene Stylization with Gaussian Splatting](https://mlsanity.com/paper/2407.07220/) (2024)

## Abstract (arXiv)

This paper studies a combination of generative Markov random field (MRF) models and discriminatively trained deep convolutional neural networks (dCNNs) for synthesizing 2D images. The generative MRF acts on higher-levels of a dCNN feature pyramid, controling the image layout at an abstract level. We apply the method to both photographic and non-photo-realistic (artwork) synthesis tasks. The MRF regularizer prevents over-excitation artifacts and reduces implausible feature mixtures common to previous dCNN inversion approaches, permitting synthezing photographic content with increased visual plausibility. Unlike standard MRF-based texture synthesis, the combined system can both match and adapt local features with considerable variability, yielding results far out of reach of classic generative MRF methods.