# Visually Similar Products Retrieval for Shopsy

arXiv: https://arxiv.org/abs/2210.04560 · HTML: https://mlsanity.com/paper/2210.04560/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 42 references that are in the map.
- First posted on arXiv 10 Oct 2022.

## Builds on (most cited first, 20 of 20)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [Billion-scale similarity search with GPUs](https://mlsanity.com/paper/1702.08734/) (2017)
- [Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics](https://mlsanity.com/paper/1705.07115/) (2017)
- [GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networks](https://mlsanity.com/paper/1711.02257/) (2017)
- [Semi-Supervised Learning with Deep Generative Models](https://mlsanity.com/paper/1406.5298/) (2014)
- [Deep Metric Learning via Lifted Structured Feature Embedding](https://mlsanity.com/paper/1511.06452/) (2015)
- [No Fuss Distance Metric Learning using Proxies](https://mlsanity.com/paper/1703.07464/) (2017)
- [Proxy Anchor Loss for Deep Metric Learning](https://mlsanity.com/paper/2003.13911/) (2020)
- [Cross-Domain Self-supervised Multi-task Feature Learning using Synthetic Imagery](https://mlsanity.com/paper/1711.09082/) (2017)
- [Visual Search at Alibaba](https://mlsanity.com/paper/2102.04674/) (2021)
- [Visual Search at eBay](https://mlsanity.com/paper/1706.03154/) (2017)
- [Learning a Unified Embedding for Visual Search at Pinterest](https://mlsanity.com/paper/1908.01707/) (2019)
- [Visual Search at Pinterest](https://mlsanity.com/paper/1505.07647/) (2015)
- [Deep Learning based Large Scale Visual Recommendation and Search for E-Commerce](https://mlsanity.com/paper/1703.02344/) (2017)
- [Web-Scale Responsive Visual Search at Bing](https://mlsanity.com/paper/1802.04914/) (2018)
- [Visual Discovery at Pinterest](https://mlsanity.com/paper/1702.04680/) (2017)
- [A Unified Model with Structured Output for Fashion Images Classification](https://mlsanity.com/paper/1806.09445/) (2018)
- [Exploiting Latent Codes: Interactive Fashion Product Generation, Similar Image Retrieval, and Cross-Category Recommendation using Variational Autoencoders](https://mlsanity.com/paper/2009.01053/) (2020)
- [FashionSearchNet-v2: Learning Attribute Representations with Localization for Image Retrieval with Attribute Manipulation](https://mlsanity.com/paper/2111.14145/) (2021)

## Built on by (most cited first, 1 of 1)

- [OneVision: An End-to-End Generative Framework for Multi-view E-commerce Vision Search](https://mlsanity.com/paper/2510.05759/) (2025)

## Abstract (arXiv)

Visual search is of great assistance in reseller commerce, especially for non-tech savvy users with affinity towards regional languages. It allows resellers to accurately locate the products that they seek, unlike textual search which recommends products from head brands. Product attributes available in e-commerce have a great potential for building better visual search systems as they capture fine grained relations between data points. In this work, we design a visual search system for reseller commerce using a multi-task learning approach. We also highlight and address the challenges like image compression, cropping, scribbling on the image, etc, faced in reseller commerce. Our model consists of three different tasks: attribute classification, triplet ranking and variational autoencoder (VAE). Masking technique is used for designing the attribute classification. Next, we introduce an offline triplet mining technique which utilizes information from multiple attributes to capture relative order within the data. This technique displays a better performance compared to the traditional triplet mining baseline, which uses single label/attribute information. We also compare and report incremental gain achieved by our unified multi-task model over each individual task separately. The effectiveness of our method is demonstrated using the in-house dataset of product images from the Lifestyle business-unit of Flipkart, India's largest e-commerce company. To efficiently retrieve the images in production, we use the Approximate Nearest Neighbor (ANN) index. Finally, we highlight our production environment constraints and present the design choices and experiments conducted to select a suitable ANN index.