# Unsupervised Robust Disentangling of Latent Characteristics for Image Synthesis

arXiv: https://arxiv.org/abs/1910.10223 · HTML: https://mlsanity.com/paper/1910.10223/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Oct 2019.

## Built on by (most cited first, 12 of 12)

- [Learning Disentangled Representations in the Imaging Domain](https://mlsanity.com/paper/2108.12043/) (2021)
- [Understanding Object Dynamics for Interactive Image-to-Video Synthesis](https://mlsanity.com/paper/2106.11303/) (2021)
- [Behavior-Driven Synthesis of Human Dynamics](https://mlsanity.com/paper/2103.04677/) (2021)
- [Flow Guided Transformable Bottleneck Networks for Motion Retargeting](https://mlsanity.com/paper/2106.07771/) (2021)
- [Style Equalization: Unsupervised Learning of Controllable Generative Sequence Models](https://mlsanity.com/paper/2110.02891/) (2021)
- [Disentanglement Analysis with Partial Information Decomposition](https://mlsanity.com/paper/2108.13753/) (2021)
- [AE-NeRF: Auto-Encoding Neural Radiance Fields for 3D-Aware Object Manipulation](https://mlsanity.com/paper/2204.13426/) (2022)
- [Generating Synthetic Satellite Imagery for Rare Objects: An Empirical Comparison of Models and Metrics](https://mlsanity.com/paper/2409.01138/) (2024)
- [Symbolic Disentangled Representations for Images](https://mlsanity.com/paper/2412.19847/) (2024)
- [Unsupervised Structure-Consistent Image-to-Image Translation](https://mlsanity.com/paper/2208.11546/) (2022)
- [Break The Spell Of Total Correlation In betaTCVAE](https://mlsanity.com/paper/2210.08794/) (2022)
- [DualVAE: Controlling Colours of Generated and Real Images](https://mlsanity.com/paper/2305.18769/) (2023)

## Abstract (arXiv)

Deep generative models come with the promise to learn an explainable representation for visual objects that allows image sampling, synthesis, and selective modification. The main challenge is to learn to properly model the independent latent characteristics of an object, especially its appearance and pose. We present a novel approach that learns disentangled representations of these characteristics and explains them individually. Training requires only pairs of images depicting the same object appearance, but no pose annotations. We propose an additional classifier that estimates the minimal amount of regularization required to enforce disentanglement. Thus both representations together can completely explain an image while being independent of each other. Previous methods based on adversarial approaches fail to enforce this independence, while methods based on variational approaches lead to uninformative representations. In experiments on diverse object categories, the approach successfully recombines pose and appearance to reconstruct and retarget novel synthesized images. We achieve significant improvements over state-of-the-art methods which utilize the same level of supervision, and reach performances comparable to those of pose-supervised approaches. However, we can handle the vast body of articulated object classes for which no pose models/annotations are available.