# Partial Visual-Semantic Embedding: Fashion Intelligence System with Sensitive Part-by-Part Learning

arXiv: https://arxiv.org/abs/2211.06688 · HTML: https://mlsanity.com/paper/2211.06688/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 31 of its 52 references that are in the map.
- First posted on arXiv 12 Nov 2022.

## Builds on (most cited first, 30 of 31)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [Deep Visual-Semantic Alignments for Generating Image Descriptions](https://mlsanity.com/paper/1412.2306/) (2014)
- [VITON: An Image-based Virtual Try-on Network](https://mlsanity.com/paper/1711.08447/) (2017)
- [Learning Fine-grained Image Similarity with Deep Ranking](https://mlsanity.com/paper/1404.4661/) (2014)
- [Self-Correction for Human Parsing](https://mlsanity.com/paper/1910.09777/) (2019)
- [ZeroCap: Zero-Shot Image-to-Text Generation for Visual-Semantic Arithmetic](https://mlsanity.com/paper/2111.14447/) (2021)
- [Exploring Models and Data for Image Question Answering](https://mlsanity.com/paper/1505.02074/) (2015)
- [Automatic Spatially-aware Fashion Concept Discovery](https://mlsanity.com/paper/1708.01311/) (2017)
- [Deep Metric Learning with Angular Loss](https://mlsanity.com/paper/1708.01682/) (2017)
- [Look into Person: Joint Body Parsing & Pose Estimation Network and A New Benchmark](https://mlsanity.com/paper/1804.01984/) (2018)
- [Learning Fashion Compatibility with Bidirectional LSTMs](https://mlsanity.com/paper/1707.05691/) (2017)
- [Devil in the Details: Towards Accurate Single and Multiple Human Parsing](https://mlsanity.com/paper/1809.05996/) (2018)
- [POG: Personalized Outfit Generation for Fashion Recommendation at Alibaba iFashion](https://mlsanity.com/paper/1905.01866/) (2019)
- [Learning Type-Aware Embeddings for Fashion Compatibility](https://mlsanity.com/paper/1803.09196/) (2018)
- [Semantics Disentangling for Text-to-Image Generation](https://mlsanity.com/paper/1904.01480/) (2019)
- [Beyond Human Parts: Dual Part-Aligned Representations for Person Re-Identification](https://mlsanity.com/paper/1910.10111/) (2019)
- [ClothFormer:Taming Video Virtual Try-on in All Module](https://mlsanity.com/paper/2204.12151/) (2022)
- [Fashion Forward: Forecasting Visual Style in Fashion](https://mlsanity.com/paper/1705.06394/) (2017)
- [Creating Capsule Wardrobes from Fashion Images](https://mlsanity.com/paper/1712.02662/) (2017)
- [Complete the Look: Scene-based Complementary Product Recommendation](https://mlsanity.com/paper/1812.01748/) (2018)
- [Fashion++: Minimal Edits for Outfit Improvement](https://mlsanity.com/paper/1904.09261/) (2019)
- [Neural Compatibility Modeling with Attentive Knowledge Distillation](https://mlsanity.com/paper/1805.00313/) (2018)
- [Mining Fashion Outfit Composition Using An End-to-End Deep Learning Approach on Set Data](https://mlsanity.com/paper/1608.03016/) (2016)
- [Single Image Action Recognition using Semantic Body Part Actions](https://mlsanity.com/paper/1612.04520/) (2016)
- [DeepStyle: Multimodal Search Engine for Fashion and Interior Design](https://mlsanity.com/paper/1801.03002/) (2018)
- [Global Weighted Average Pooling Bridges Pixel-level Localization and Image-level Classification](https://mlsanity.com/paper/1809.08264/) (2018)
- [Multigranular Visual-Semantic Embedding for Cloth-Changing Person Re-identification](https://mlsanity.com/paper/2108.04527/) (2021)
- [Explaining Deep Convolutional Neural Networks via Latent Visual-Semantic Filter Attention](https://mlsanity.com/paper/2204.04601/) (2022)
- [T-VSE: Transformer-Based Visual Semantic Embedding](https://mlsanity.com/paper/2005.08399/) (2020)

## Abstract (arXiv)

In this study, we propose a technology called the Fashion Intelligence System based on the visual-semantic embedding (VSE) model to quantify abstract and complex expressions unique to fashion, such as ''casual,'' ''adult-casual,'' and ''office-casual,'' and to support users' understanding of fashion. However, the existing VSE model does not support the situations in which the image is composed of multiple parts such as hair, tops, pants, skirts, and shoes. We propose partial VSE, which enables sensitive learning for each part of the fashion coordinates. The proposed model partially learns embedded representations. This helps retain the various existing practical functionalities and enables image-retrieval tasks in which changes are made only to the specified parts and image reordering tasks that focus on the specified parts. This was not possible with conventional models. Based on both the qualitative and quantitative evaluation experiments, we show that the proposed model is superior to conventional models without increasing the computational complexity.