Generating Furry Cars: Disentangling Object Shape & Appearance across Multiple Domains
Not on the map. Why?
Abstract (arXiv)
We consider the novel task of learning disentangled representations of object shape and appearance across multiple domains (e.g., dogs and cars). The goal is to learn a generative model that learns an intermediate distribution, which borrows a subset of properties from each domain, enabling the generation of images that did not exist in any domain exclusively. This challenging problem requires an accurate disentanglement of object shape, appearance, and background from each domain, so that the appearance and shape factors from the two domains can be interchanged. We augment an existing approach that can disentangle factors within a single domain but struggles to do so across domains. Our key technical contribution is to represent object appearance with a differentiable histogram of visual features, and to optimize the generator so that two images with the same latent appearance factor but different latent shape factors produce similar histograms. On multiple multi-domain datasets, we demonstrate our method leads to accurate and consistent appearance and shape transfer across domains.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 29 of its 40 references that are in the map. First posted on arXiv 5 Apr 2021.
Builds on
29 of 40 references.
References · 29
- 2014Microsoft COCO: Common Objects in Contextcited by 17,498
- 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
- 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
- 2013Auto-Encoding Variational Bayescited by 3,980
- 2016Perceptual Losses for Real-Time Style Transfer and Super-Resolutioncited by 2,678
- 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
- 2017Rethinking Atrous Convolution for Semantic Image Segmentationcited by 2,154
- 2015Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networkscited by 2,048
- 2017Arbitrary Style Transfer in Real-time with Adaptive Instance Normalizationcited by 1,998
- 2019StarGAN v2: Diverse Image Synthesis for Multiple Domainscited by 1,095
- 2016InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Netscited by 858
- 2018Multimodal Unsupervised Image-to-Image Translationcited by 795
17 more
- 2019Learning Representations by Maximizing Mutual Information Across Viewscited by 733
- 2015A Neural Algorithm of Artistic Stylecited by 480
- 2017Learning to Discover Cross-Domain Relations with Generative Adversarial Networkscited by 278
- 2019DRIT++: Diverse Image-to-Image Translation via Disentangled Representationscited by 244
- 2017StackGAN++: Realistic Image Synthesis with Stacked Generative Adversarial Networkscited by 229
- 2019Few-Shot Unsupervised Image-to-Image Translationcited by 222
- 2016Texture Networks: Feed-forward Synthesis of Textures and Stylized Imagescited by 173
- 2017Unsupervised Learning of Disentangled Representations from Videocited by 138
- 2017Disentangled Person Image Generationcited by 98
- 2018Synthesizing Images of Humans in Unseen Posescited by 96
- 2018Image-to-image translation for cross-domain disentanglementcited by 61
- 2018Deforming Autoencoders: Unsupervised Disentangling of Shape and Appearancecited by 58
- 2018FineGAN: Unsupervised Hierarchical Disentanglement for Fine-Grained Object Generation and Discoverycited by 58
- 2019MixNMatch: Multifactor Disentanglement and Encoding for Conditional Image Generationcited by 29
- 2017Reconstruction-Based Disentanglement for Pose-invariant Face Recognitioncited by 19
- 2017Disentangling Factors of Variation by Mixing Themcited by 18
- 2018Unsupervised Disentangled Representation Learning with Analogical Relationscited by 2
Built on by
Citing papers per year: 2021: 1; 2022: 1; 2023: 1.
3 papers.