# Semi-parametric Image Synthesis

arXiv: https://arxiv.org/abs/1804.10992 · HTML: https://mlsanity.com/paper/1804.10992/

## Key facts

- Cited by 50 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Apr 2018.

## Built on by (most cited first, 30 of 50)

- [TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Driving](https://mlsanity.com/paper/2205.15997/) (2022)
- [Multi-Modal Fusion Transformer for End-to-End Autonomous Driving](https://mlsanity.com/paper/2104.09224/) (2021)
- [Semantic Image Synthesis via Diffusion Models](https://mlsanity.com/paper/2207.00050/) (2022)
- [Instance-Conditioned GAN](https://mlsanity.com/paper/2109.05070/) (2021)
- [GeoSim: Realistic Video Simulation via Geometry-Aware Composition for Self-Driving](https://mlsanity.com/paper/2101.06543/) (2021)
- [Enhancing Photorealism Enhancement](https://mlsanity.com/paper/2105.04619/) (2021)
- [Freestyle Layout-to-Image Synthesis](https://mlsanity.com/paper/2303.14412/) (2023)
- [ImageBrush: Learning Visual In-Context Instructions for Exemplar-Based Image Manipulation](https://mlsanity.com/paper/2308.00906/) (2023)
- [SemanticStyleGAN: Learning Compositional Generative Priors for Controllable Image Synthesis and Editing](https://mlsanity.com/paper/2112.02236/) (2021)
- [Multimodal Image Synthesis and Editing: The Generative AI Era](https://mlsanity.com/paper/2112.13592/) (2021)
- [Diverse Semantic Image Synthesis via Probability Distribution Modeling](https://mlsanity.com/paper/2103.06878/) (2021)
- [Synthetic Datasets for Autonomous Driving: A Survey](https://mlsanity.com/paper/2304.12205/) (2023)
- [Image Synthesis via Semantic Composition](https://mlsanity.com/paper/2109.07053/) (2021)
- [Guided Co-Modulated GAN for 360deg Field of View Extrapolation](https://mlsanity.com/paper/2204.07286/) (2022)
- [Retrieving-to-Answer: Zero-Shot Video Question Answering with Frozen Large Language Models](https://mlsanity.com/paper/2306.11732/) (2023)
- [MIDMs: Matching Interleaved Diffusion Models for Exemplar-based Image Translation](https://mlsanity.com/paper/2209.11047/) (2022)
- [PLACE: Adaptive Layout-Semantic Fusion for Semantic Image Synthesis](https://mlsanity.com/paper/2403.01852/) (2024)
- [Semantic-shape Adaptive Feature Modulation for Semantic Image Synthesis](https://mlsanity.com/paper/2203.16898/) (2022)
- [Retrieval-based Spatially Adaptive Normalization for Semantic Image Synthesis](https://mlsanity.com/paper/2204.02854/) (2022)
- [SinIR: Efficient General Image Manipulation with Single Image Reconstruction](https://mlsanity.com/paper/2106.07140/) (2021)
- [Memory-Driven Text-to-Image Generation](https://mlsanity.com/paper/2208.07022/) (2022)
- [A multi-stage GAN for multi-organ chest X-ray image generation and segmentation](https://mlsanity.com/paper/2106.05132/) (2021)
- [Deep Image Synthesis from Intuitive User Input: A Review and Perspectives](https://mlsanity.com/paper/2107.04240/) (2021)
- [Layout-to-Image Translation with Double Pooling Generative Adversarial Networks](https://mlsanity.com/paper/2108.12900/) (2021)
- [USIS: Unsupervised Semantic Image Synthesis](https://mlsanity.com/paper/2109.14715/) (2021)
- [Local and Global GANs with Semantic-Aware Upsampling for Image Generation](https://mlsanity.com/paper/2203.00047/) (2022)
- [Edge Guided GANs with Multi-Scale Contrastive Learning for Semantic Image Synthesis](https://mlsanity.com/paper/2307.12084/) (2023)
- [PatchRD: Detail-Preserving Shape Completion by Learning Patch Retrieval and Deformation](https://mlsanity.com/paper/2207.11790/) (2022)
- [Dual Pyramid Generative Adversarial Networks for Semantic Image Synthesis](https://mlsanity.com/paper/2210.04085/) (2022)
- [SCP-Diff: Spatial-Categorical Joint Prior for Diffusion Based Semantic Image Synthesis](https://mlsanity.com/paper/2403.09638/) (2024)

## Abstract (arXiv)

We present a semi-parametric approach to photographic image synthesis from semantic layouts. The approach combines the complementary strengths of parametric and nonparametric techniques. The nonparametric component is a memory bank of image segments constructed from a training set of images. Given a novel semantic layout at test time, the memory bank is used to retrieve photographic references that are provided as source material to a deep network. The synthesis is performed by a deep network that draws on the provided photographic material. Experiments on multiple semantic segmentation datasets show that the presented approach yields considerably more realistic images than recent purely parametric techniques. The results are shown in the supplementary video at https://youtu.be/U4Q98lenGLQ