# MixSyn: Learning Composition and Style for Multi-Source Image Synthesis

arXiv: https://arxiv.org/abs/2111.12705 · HTML: https://mlsanity.com/paper/2111.12705/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 50 references that are in the map.
- First posted on arXiv 24 Nov 2021.

## Builds on (most cited first, 30 of 35)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization](https://mlsanity.com/paper/1703.06868/) (2017)
- [Identity Mappings in Deep Residual Networks](https://mlsanity.com/paper/1603.05027/) (2016)
- [Spectral Normalization for Generative Adversarial Networks](https://mlsanity.com/paper/1802.05957/) (2018)
- [Semantic Image Synthesis with Spatially-Adaptive Normalization](https://mlsanity.com/paper/1903.07291/) (2019)
- [High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs](https://mlsanity.com/paper/1711.11585/) (2017)
- [StarGAN v2: Diverse Image Synthesis for Multiple Domains](https://mlsanity.com/paper/1912.01865/) (2019)
- [StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translation](https://mlsanity.com/paper/1711.09020/) (2017)
- [Which Training Methods for GANs do actually Converge?](https://mlsanity.com/paper/1801.04406/) (2018)
- [Contrastive Learning for Unpaired Image-to-Image Translation](https://mlsanity.com/paper/2007.15651/) (2020)
- [MaskGAN: Towards Diverse and Interactive Facial Image Manipulation](https://mlsanity.com/paper/1907.11922/) (2019)
- [SEAN: Image Synthesis with Semantic Region-Adaptive Normalization](https://mlsanity.com/paper/1911.12861/) (2019)
- [Fake It Till You Make It: Face analysis in the wild using synthetic data alone](https://mlsanity.com/paper/2109.15102/) (2021)
- [Editing in Style: Uncovering the Local Semantics of GANs](https://mlsanity.com/paper/2004.14367/) (2020)
- [Semantic Photo Manipulation with a Generative Image Prior](https://mlsanity.com/paper/2005.07727/) (2020)
- [Semantic Image Manipulation Using Scene Graphs](https://mlsanity.com/paper/2004.03677/) (2020)
- [Transforming and Projecting Images into Class-conditional Generative Networks](https://mlsanity.com/paper/2005.01703/) (2020)
- [Local Class-Specific and Global Image-Level Generative Adversarial Networks for Semantic-Guided Scene Generation](https://mlsanity.com/paper/1912.12215/) (2019)
- [Mask-Guided Portrait Editing with Conditional GANs](https://mlsanity.com/paper/1905.10346/) (2019)
- [Using latent space regression to analyze and leverage compositionality in GANs](https://mlsanity.com/paper/2103.10426/) (2021)
- [Controllable Artistic Text Style Transfer via Shape-Matching GAN](https://mlsanity.com/paper/1905.01354/) (2019)
- [SESAME: Semantic Editing of Scenes by Adding, Manipulating or Erasing Objects](https://mlsanity.com/paper/2004.04977/) (2020)
- [RetrieveGAN: Image Synthesis via Differentiable Patch Retrieval](https://mlsanity.com/paper/2007.08513/) (2020)
- [Face Parsing with RoI Tanh-Warping](https://mlsanity.com/paper/1906.01342/) (2019)
- [Deep Generation of Face Images from Sketches](https://mlsanity.com/paper/2006.01047/) (2020)
- [This Person (Probably) Exists. Identity Membership Attacks Against GAN Generated Faces](https://mlsanity.com/paper/2107.06018/) (2021)
- [Spatially Controllable Image Synthesis with Internal Representation Collaging](https://mlsanity.com/paper/1811.10153/) (2018)
- [CAFE-GAN: Arbitrary Face Attribute Editing with Complementary Attention Feature](https://mlsanity.com/paper/2011.11900/) (2020)
- [MichiGAN: Multi-Input-Conditioned Hair Image Generation for Portrait Editing](https://mlsanity.com/paper/2010.16417/) (2020)

## Built on by (most cited first, 2 of 2)

- [My Face My Choice: Privacy Enhancing Deepfakes for Social Media Anonymization](https://mlsanity.com/paper/2211.01361/) (2022)
- [How Do Deepfakes Move? Motion Magnification for Deepfake Source Detection](https://mlsanity.com/paper/2212.14033/) (2022)

## Abstract (arXiv)

Synthetic images created by generative models increase in quality and expressiveness as newer models utilize larger datasets and novel architectures. Although this photorealism is a positive side-effect from a creative standpoint, it becomes problematic when such generative models are used for impersonation without consent. Most of these approaches are built on the partial transfer between source and target pairs, or they generate completely new samples based on an ideal distribution, still resembling the closest real sample in the dataset. We propose MixSyn (read as " mixin' ") for learning novel fuzzy compositions from multiple sources and creating novel images as a mix of image regions corresponding to the compositions. MixSyn not only combines uncorrelated regions from multiple source masks into a coherent semantic composition, but also generates mask-aware high quality reconstructions of non-existing images. We compare MixSyn to state-of-the-art single-source sequential generation and collage generation approaches in terms of quality, diversity, realism, and expressive power; while also showcasing interactive synthesis, mix & match, and edit propagation tasks, with no mask dependency.