# FashionSearchNet-v2: Learning Attribute Representations with Localization for Image Retrieval with Attribute Manipulation

arXiv: https://arxiv.org/abs/2111.14145 · HTML: https://mlsanity.com/paper/2111.14145/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 50 of its 99 references that are in the map.
- First posted on arXiv 28 Nov 2021.

## Builds on (most cited first, 30 of 50)

- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [Fast R-CNN](https://mlsanity.com/paper/1504.08083/) (2015)
- [Learning Deep Features for Discriminative Localization](https://mlsanity.com/paper/1512.04150/) (2015)
- [Spatial Transformer Networks](https://mlsanity.com/paper/1506.02025/) (2015)
- [StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translation](https://mlsanity.com/paper/1711.09020/) (2017)
- [Deep metric learning using Triplet network](https://mlsanity.com/paper/1412.6622/) (2014)
- [PixelCNN++: Improving the PixelCNN with Discretized Logistic Mixture Likelihood and Other Modifications](https://mlsanity.com/paper/1701.05517/) (2017)
- [VITON: An Image-based Virtual Try-on Network](https://mlsanity.com/paper/1711.08447/) (2017)
- [Composing Text and Image for Image Retrieval - An Empirical Odyssey](https://mlsanity.com/paper/1812.07119/) (2018)
- [Object Region Mining with Adversarial Erasing: A Simple Classification to Semantic Segmentation Approach](https://mlsanity.com/paper/1703.08448/) (2017)
- [Deep Image Retrieval: Learning global representations for image search](https://mlsanity.com/paper/1604.01325/) (2016)
- [Automatic Spatially-aware Fashion Concept Discovery](https://mlsanity.com/paper/1708.01311/) (2017)
- [Learning Fashion Compatibility with Bidirectional LSTMs](https://mlsanity.com/paper/1707.05691/) (2017)
- [Part-Stacked CNN for Fine-Grained Visual Categorization](https://mlsanity.com/paper/1512.08086/) (2015)
- [Fashionpedia: Ontology, Segmentation, and an Attribute Localization Dataset](https://mlsanity.com/paper/2004.12276/) (2020)
- [Deep Imbalanced Attribute Classification using Visual Attention Aggregation](https://mlsanity.com/paper/1807.03903/) (2018)
- [Dressing in Order: Recurrent Person Image Generation for Pose Transfer, Virtual Try-on and Outfit Editing](https://mlsanity.com/paper/2104.07021/) (2021)
- [Cross-domain Image Retrieval with a Dual Attribute-aware Ranking Network](https://mlsanity.com/paper/1505.07922/) (2015)
- [Pedestrian Attribute Recognition: A Survey](https://mlsanity.com/paper/1901.07474/) (2019)
- [Be Your Own Prada: Fashion Synthesis with Structural Coherence](https://mlsanity.com/paper/1710.07346/) (2017)
- [Improving Pedestrian Attribute Recognition With Weakly-Supervised Multi-Scale Attribute-Specific Localization](https://mlsanity.com/paper/1910.04562/) (2019)
- [Fashion Outfit Complementary Item Retrieval](https://mlsanity.com/paper/1912.08967/) (2019)
- [The iMaterialist Fashion Attribute Dataset](https://mlsanity.com/paper/1906.05750/) (2019)
- [Learning Visual Clothing Style with Heterogeneous Dyadic Co-occurrences](https://mlsanity.com/paper/1509.07473/) (2015)
- [Fashion Retrieval via Graph Reasoning Networks on a Similarity Pyramid](https://mlsanity.com/paper/1908.11754/) (2019)
- [Video2Shop: Exact Matching Clothes in Videos to Online Shopping Images](https://mlsanity.com/paper/1804.05287/) (2018)
- [Compatible and Diverse Fashion Image Inpainting](https://mlsanity.com/paper/1902.01096/) (2019)
- [Fashion Forward: Forecasting Visual Style in Fashion](https://mlsanity.com/paper/1705.06394/) (2017)

## Built on by (most cited first, 2 of 2)

- [Supervised Attribute Information Removal and Reconstruction for Image Manipulation](https://mlsanity.com/paper/2207.06555/) (2022)
- [Visually Similar Products Retrieval for Shopsy](https://mlsanity.com/paper/2210.04560/) (2022)

## Abstract (arXiv)

The focus of this paper is on the problem of image retrieval with attribute manipulation. Our proposed work is able to manipulate the desired attributes of the query image while maintaining its other attributes. For example, the collar attribute of the query image can be changed from round to v-neck to retrieve similar images from a large dataset. A key challenge in e-commerce is that images have multiple attributes where users would like to manipulate and it is important to estimate discriminative feature representations for each of these attributes. The proposed FashionSearchNet-v2 architecture is able to learn attribute specific representations by leveraging on its weakly-supervised localization module, which ignores the unrelated features of attributes in the feature space, thus improving the similarity learning. The network is jointly trained with the combination of attribute classification and triplet ranking loss to estimate local representations. These local representations are then merged into a single global representation based on the instructed attribute manipulation where desired images can be retrieved with a distance metric. The proposed method also provides explainability for its retrieval process to help provide additional information on the attention of the network. Experiments performed on several datasets that are rich in terms of the number of attributes show that FashionSearchNet-v2 outperforms the other state-of-the-art attribute manipulation techniques. Different than our earlier work (FashionSearchNet), we propose several improvements in the learning procedure and show that the proposed FashionSearchNet-v2 can be generalized to different domains other than fashion.