# Image-to-image Translation via Hierarchical Style Disentanglement

arXiv: https://arxiv.org/abs/2103.01456 · HTML: https://mlsanity.com/paper/2103.01456/

## Key facts

- Cited by 63 papers in the map (data through 14 Oct 2026).
- Builds on 50 of its 57 references that are in the map.
- First posted on arXiv 2 Mar 2021.

## Builds on (most cited first, 30 of 50)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [Improved Training of Wasserstein GANs](https://mlsanity.com/paper/1704.00028/) (2017)
- [Conditional Generative Adversarial Nets](https://mlsanity.com/paper/1411.1784/) (2014)
- [Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization](https://mlsanity.com/paper/1703.06868/) (2017)
- [Identity Mappings in Deep Residual Networks](https://mlsanity.com/paper/1603.05027/) (2016)
- [Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network](https://mlsanity.com/paper/1609.04802/) (2016)
- [Spectral Normalization for Generative Adversarial Networks](https://mlsanity.com/paper/1802.05957/) (2018)
- [Instance Normalization: The Missing Ingredient for Fast Stylization](https://mlsanity.com/paper/1607.08022/) (2016)
- [Colorful Image Colorization](https://mlsanity.com/paper/1603.08511/) (2016)
- [CyCADA: Cycle-Consistent Adversarial Domain Adaptation](https://mlsanity.com/paper/1711.03213/) (2017)
- [StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translation](https://mlsanity.com/paper/1711.09020/) (2017)
- [Least Squares Generative Adversarial Networks](https://mlsanity.com/paper/1611.04076/) (2016)
- [Multimodal Unsupervised Image-to-Image Translation](https://mlsanity.com/paper/1804.04732/) (2018)
- [Generative Adversarial Text to Image Synthesis](https://mlsanity.com/paper/1605.05396/) (2016)
- [Which Training Methods for GANs do actually Converge?](https://mlsanity.com/paper/1801.04406/) (2018)
- [Unsupervised Image-to-Image Translation Networks](https://mlsanity.com/paper/1703.00848/) (2017)
- [InterFaceGAN: Interpreting the Disentangled Face Representation Learned by GANs](https://mlsanity.com/paper/2005.09635/) (2020)
- [DualGAN: Unsupervised Dual Learning for Image-to-Image Translation](https://mlsanity.com/paper/1704.02510/) (2017)
- [Toward Multimodal Image-to-Image Translation](https://mlsanity.com/paper/1711.11586/) (2017)
- [DRIT++: Diverse Image-to-Image Translation via Disentangled Representations](https://mlsanity.com/paper/1905.01270/) (2019)
- [AttGAN: Facial Attribute Editing by Only Changing What You Want](https://mlsanity.com/paper/1711.10678/) (2017)
- [Photographic Image Synthesis with Cascaded Refinement Networks](https://mlsanity.com/paper/1707.09405/) (2017)
- [Few-Shot Unsupervised Image-to-Image Translation](https://mlsanity.com/paper/1905.01723/) (2019)
- [GANimation: Anatomically-aware Facial Animation from a Single Image](https://mlsanity.com/paper/1807.09251/) (2018)
- [Real-Time Adaptive Image Compression](https://mlsanity.com/paper/1705.05823/) (2017)

## Built on by (most cited first, 30 of 63)

- [Hierarchical Fine-Grained Image Forgery Detection and Localization](https://mlsanity.com/paper/2303.17111/) (2023)
- [Disentangled Representation Learning](https://mlsanity.com/paper/2211.11695/) (2022)
- [Talk-to-Edit: Fine-Grained Facial Editing via Dialog](https://mlsanity.com/paper/2109.04425/) (2021)
- [CMUA-Watermark: A Cross-Model Universal Adversarial Watermark for Combating Deepfakes](https://mlsanity.com/paper/2105.10872/) (2021)
- [Edge-Cloud Polarization and Collaboration: A Comprehensive Survey for AI](https://mlsanity.com/paper/2111.06061/) (2021)
- [Wavelet Knowledge Distillation: Towards Efficient Image-to-Image Translation](https://mlsanity.com/paper/2203.06321/) (2022)
- [Controllable 3D Face Synthesis with Conditional Generative Occupancy Fields](https://mlsanity.com/paper/2206.08361/) (2022)
- [Predict, Prevent, and Evaluate: Disentangled Text-Driven Image Manipulation Empowered by Pre-Trained Vision-Language Model](https://mlsanity.com/paper/2111.13333/) (2021)
- [Diverse Inpainting and Editing with GAN Inversion](https://mlsanity.com/paper/2307.15033/) (2023)
- [Detecting Generated Images by Real Images Only](https://mlsanity.com/paper/2311.00962/) (2023)
- [Interactive Image Synthesis with Panoptic Layout Generation](https://mlsanity.com/paper/2203.02104/) (2022)
- [VecGAN: Image-to-Image Translation with Interpretable Latent Directions](https://mlsanity.com/paper/2207.03411/) (2022)
- [GAN-based Facial Attribute Manipulation](https://mlsanity.com/paper/2210.12683/) (2022)
- [Language-guided Hierarchical Fine-grained Image Forgery Detection and Localization](https://mlsanity.com/paper/2410.23556/) (2024)
- [Image-to-Image Translation with Disentangled Latent Vectors for Face Editing](https://mlsanity.com/paper/2301.04628/) (2023)
- [AVA: Inconspicuous Attribute Variation-based Adversarial Attack bypassing DeepFake Detection](https://mlsanity.com/paper/2312.08675/) (2023)
- [Portrait Eyeglasses and Shadow Removal by Leveraging 3D Synthetic Data](https://mlsanity.com/paper/2203.10474/) (2022)
- [Region-aware Knowledge Distillation for Efficient Image-to-Image Translation](https://mlsanity.com/paper/2205.12451/) (2022)
- [Text-guided Eyeglasses Manipulation with Spatial Constraints](https://mlsanity.com/paper/2304.12539/) (2023)
- [Graph-based Unsupervised Disentangled Representation Learning via Multimodal Large Language Models](https://mlsanity.com/paper/2407.18999/) (2024)
- [Physics-informed Guided Disentanglement in Generative Networks](https://mlsanity.com/paper/2107.14229/) (2021)
- [Information-containing Adversarial Perturbation for Combating Facial Manipulation Systems](https://mlsanity.com/paper/2303.11625/) (2023)
- [Conditioning Diffusion Models via Attributes and Semantic Masks for Face Generation](https://mlsanity.com/paper/2306.00914/) (2023)
- [Reference-Based 3D-Aware Image Editing with Triplanes](https://mlsanity.com/paper/2404.03632/) (2024)
- [FaceShield: Defending Facial Image against Deepfake Threats](https://mlsanity.com/paper/2412.09921/) (2024)
- [DyStyle: Dynamic Neural Network for Multi-Attribute-Conditioned Style Editing](https://mlsanity.com/paper/2109.10737/) (2021)
- [Exploring Negatives in Contrastive Learning for Unpaired Image-to-Image Translation](https://mlsanity.com/paper/2204.11018/) (2022)
- [A Survey of Methods, Challenges and Perspectives in Causality](https://mlsanity.com/paper/2302.00293/) (2023)
- [Warping the Residuals for Image Editing with StyleGAN](https://mlsanity.com/paper/2312.11422/) (2023)
- [FACEMUG: A Multimodal Generative and Fusion Framework for Local Facial Editing](https://mlsanity.com/paper/2412.19009/) (2024)

## Abstract (arXiv)

Recently, image-to-image translation has made significant progress in achieving both multi-label (ie, translation conditioned on different labels) and multi-style (ie, generation with diverse styles) tasks. However, due to the unexplored independence and exclusiveness in the labels, existing endeavors are defeated by involving uncontrolled manipulations to the translation results. In this paper, we propose Hierarchical Style Disentanglement (HiSD) to address this issue. Specifically, we organize the labels into a hierarchical tree structure, in which independent tags, exclusive attributes, and disentangled styles are allocated from top to bottom. Correspondingly, a new translation process is designed to adapt the above structure, in which the styles are identified for controllable translations. Both qualitative and quantitative results on the CelebA-HQ dataset verify the ability of the proposed HiSD. We hope our method will serve as a solid baseline and provide fresh insights with the hierarchically organized annotations for future research in image-to-image translation. The code has been released at https://github.com/imlixinyang/HiSD.