# Hierarchical Conditional Semi-Paired Image-to-Image Translation For Multi-Task Image Defect Correction On Shopping Websites

arXiv: https://arxiv.org/abs/2309.05883 · HTML: https://mlsanity.com/paper/2309.05883/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 23 of its 28 references that are in the map.
- First posted on arXiv 12 Sep 2023.

## Builds on (most cited first, 23 of 23)

- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Semantic Image Synthesis with Spatially-Adaptive Normalization](https://mlsanity.com/paper/1903.07291/) (2019)
- [Self-Attention Generative Adversarial Networks](https://mlsanity.com/paper/1805.08318/) (2018)
- [Contrastive Learning for Unpaired Image-to-Image Translation](https://mlsanity.com/paper/2007.15651/) (2020)
- [Generative Image Inpainting with Contextual Attention](https://mlsanity.com/paper/1801.07892/) (2018)
- [Generative Visual Manipulation on the Natural Image Manifold](https://mlsanity.com/paper/1609.03552/) (2016)
- [Learning to Discover Cross-Domain Relations with Generative Adversarial Networks](https://mlsanity.com/paper/1703.05192/) (2017)
- [SEAN: Image Synthesis with Semantic Region-Adaptive Normalization](https://mlsanity.com/paper/1911.12861/) (2019)
- [The relativistic discriminator: a key element missing from standard GAN](https://mlsanity.com/paper/1807.00734/) (2018)
- [U-GAT-IT: Unsupervised Generative Attentional Networks with Adaptive Layer-Instance Normalization for Image-to-Image Translation](https://mlsanity.com/paper/1907.10830/) (2019)
- [Semantic Segmentation with Generative Models: Semi-Supervised Learning and Strong Out-of-Domain Generalization](https://mlsanity.com/paper/2104.05833/) (2021)
- [AttentionGAN: Unpaired Image-to-Image Translation using Attention-Guided Generative Adversarial Networks](https://mlsanity.com/paper/1911.11897/) (2019)
- [Cross-View Image Synthesis using Conditional GANs](https://mlsanity.com/paper/1803.03396/) (2018)
- [Variational Approaches for Auto-Encoding Generative Adversarial Networks](https://mlsanity.com/paper/1706.04987/) (2017)
- [QS-Attn: Query-Selected Attention for Contrastive Learning in I2I Translation](https://mlsanity.com/paper/2203.08483/) (2022)
- [InstaGAN: Instance-aware Image-to-Image Translation](https://mlsanity.com/paper/1812.10889/) (2018)
- [TRACER: Extreme Attention Guided Salient Object Tracing Network](https://mlsanity.com/paper/2112.07380/) (2021)
- [InstaFormer: Instance-Aware Image-to-Image Translation with Transformer](https://mlsanity.com/paper/2203.16248/) (2022)
- [Modulated Contrast for Versatile Image Synthesis](https://mlsanity.com/paper/2203.09333/) (2022)
- [Fine-Grained Fashion Similarity Learning by Attribute-Specific Embedding Network](https://mlsanity.com/paper/2002.02814/) (2020)
- [Learning image-to-image translation using paired and unpaired training samples](https://mlsanity.com/paper/1805.03189/) (2018)
- [Learning to Transfer: Unsupervised Meta Domain Translation](https://mlsanity.com/paper/1906.00181/) (2019)
- [Extremely Weak Supervised Image-to-Image Translation for Semantic Segmentation](https://mlsanity.com/paper/1909.08542/) (2019)

## Abstract (arXiv)

On shopping websites, product images of low quality negatively affect customer experience. Although there are plenty of work in detecting images with different defects, few efforts have been dedicated to correct those defects at scale. A major challenge is that there are thousands of product types and each has specific defects, therefore building defect specific models is unscalable. In this paper, we propose a unified Image-to-Image (I2I) translation model to correct multiple defects across different product types. Our model leverages an attention mechanism to hierarchically incorporate high-level defect groups and specific defect types to guide the network to focus on defect-related image regions. Evaluated on eight public datasets, our model reduces the Frechet Inception Distance (FID) by 24.6% in average compared with MoNCE, the state-of-the-art I2I method. Unlike public data, another practical challenge on shopping websites is that some paired images are of low quality. Therefore we design our model to be semi-paired by combining the L1 loss of paired data with the cycle loss of unpaired data. Tested on a shopping website dataset to correct three image defects, our model reduces (FID) by 63.2% in average compared with WS-I2I, the state-of-the art semi-paired I2I method.