# Large Scale Long-tailed Product Recognition System at Alibaba

arXiv: https://arxiv.org/abs/2102.04652 · HTML: https://mlsanity.com/paper/2102.04652/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 36 references that are in the map.
- First posted on arXiv 9 Feb 2021.

## Builds on (most cited first, 15 of 15)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [The Open Images Dataset V4: Unified image classification, object detection, and visual relationship detection at scale](https://mlsanity.com/paper/1811.00982/) (2018)
- [Unifying distillation and privileged information](https://mlsanity.com/paper/1511.03643/) (2015)
- [Cost Sensitive Learning of Deep Feature Representations from Imbalanced Data](https://mlsanity.com/paper/1508.03422/) (2015)
- [Learning Visual Features from Large Weakly Supervised Data](https://mlsanity.com/paper/1511.02251/) (2015)
- [Range Loss for Deep Face Recognition with Long-tail](https://mlsanity.com/paper/1611.08976/) (2016)
- [Class Rectification Hard Mining for Imbalanced Deep Learning](https://mlsanity.com/paper/1712.03162/) (2017)
- [Imbalanced Deep Learning by Minority Class Incremental Rectification](https://mlsanity.com/paper/1804.10851/) (2018)
- [Person Re-identification with Metric Learning using Privileged Information](https://mlsanity.com/paper/1904.05005/) (2019)
- [Leveraging Weakly Annotated Data for Fashion Image Retrieval and Label Prediction](https://mlsanity.com/paper/1709.09426/) (2017)
- [Who, Where, and What to Wear? Extracting Fashion Knowledge from Social Media](https://mlsanity.com/paper/1908.08985/) (2019)
- [WebVision Challenge: Visual Learning and Understanding With Web Data](https://mlsanity.com/paper/1705.05640/) (2017)
- [Everything old is new again: A multi-view learning approach to learning using privileged information and distillation](https://mlsanity.com/paper/1903.03694/) (2019)

## Built on by (most cited first, 3 of 3)

- [Billion-Scale Pretraining with Vision Transformers for Multi-Task Visual Representations](https://mlsanity.com/paper/2108.05887/) (2021)
- [CAPro: Webly Supervised Learning with Cross-Modality Aligned Prototypes](https://mlsanity.com/paper/2310.09761/) (2023)
- [FoPro: Few-Shot Guided Robust Webly-Supervised Prototypical Learning](https://mlsanity.com/paper/2212.00465/) (2022)

## Abstract (arXiv)

A practical large scale product recognition system suffers from the phenomenon of long-tailed imbalanced training data under the E-commercial circumstance at Alibaba. Besides product images at Alibaba, plenty of image related side information (e.g. title, tags) reveal rich semantic information about images. Prior works mainly focus on addressing the long tail problem in visual perspective only, but lack of consideration of leveraging the side information. In this paper, we present a novel side information based large scale visual recognition co-training~(SICoT) system to deal with the long tail problem by leveraging the image related side information. In the proposed co-training system, we firstly introduce a bilinear word attention module aiming to construct a semantic embedding over the noisy side information. A visual feature and semantic embedding co-training scheme is then designed to transfer knowledge from classes with abundant training data (head classes) to classes with few training data (tail classes) in an end-to-end fashion. Extensive experiments on four challenging large scale datasets, whose numbers of classes range from one thousand to one million, demonstrate the scalable effectiveness of the proposed SICoT system in alleviating the long tail problem. In the visual search platform Pailitaofootnotehttp://www.pailitao.com at Alibaba, we settle a practical large scale product recognition application driven by the proposed SICoT system, and achieve a significant gain of unique visitor~(UV) conversion rate.