# V2L: Leveraging Vision and Vision-language Models into Large-scale Product Retrieval

arXiv: https://arxiv.org/abs/2207.12994 · HTML: https://mlsanity.com/paper/2207.12994/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Document understanding and multimodal RAG, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 24 of its 26 references that are in the map.
- First posted on arXiv 26 Jul 2022.

## Placed in

- Multimodal LLMs / vision-language models / [Document understanding and multimodal RAG](https://mlsanity.com/mllm/document-understanding-and-multimodal-rag/)

## Builds on (most cited first, 24 of 24)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation](https://mlsanity.com/paper/2201.12086/) (2022)
- [Aggregated Residual Transformations for Deep Neural Networks](https://mlsanity.com/paper/1611.05431/) (2016)
- [BEiT: BERT Pre-Training of Image Transformers](https://mlsanity.com/paper/2106.08254/) (2021)
- [Align before Fuse: Vision and Language Representation Learning with Momentum Distillation](https://mlsanity.com/paper/2107.07651/) (2021)
- [SimMIM: A Simple Framework for Masked Image Modeling](https://mlsanity.com/paper/2111.09886/) (2021)
- [CosFace: Large Margin Cosine Loss for Deep Face Recognition](https://mlsanity.com/paper/1801.09414/) (2018)
- [In Defense of the Triplet Loss for Person Re-Identification](https://mlsanity.com/paper/1703.07737/) (2017)
- [Fine-tuning CNN Image Retrieval with No Human Annotation](https://mlsanity.com/paper/1711.02512/) (2017)
- [ResNeSt: Split-Attention Networks](https://mlsanity.com/paper/2004.08955/) (2020)
- [SLIP: Self-supervision meets Language-Image Pre-training](https://mlsanity.com/paper/2112.12750/) (2021)
- [An Empirical Study of Training End-to-End Vision-and-Language Transformers](https://mlsanity.com/paper/2111.02387/) (2021)
- [Re-ranking Person Re-identification with k-reciprocal Encoding](https://mlsanity.com/paper/1701.08398/) (2017)
- [Multi-Grained Vision Language Pre-Training: Aligning Texts with Visual Concepts](https://mlsanity.com/paper/2111.08276/) (2021)
- [Neighborhood Attention Transformer](https://mlsanity.com/paper/2204.07143/) (2022)
- [Contextual Transformer Networks for Visual Recognition](https://mlsanity.com/paper/2107.12292/) (2021)
- [Products-10K: A Large-scale Product Recognition Dataset](https://mlsanity.com/paper/2008.10545/) (2020)
- [Weakly Supervised Learning with Side Information for Noisy Labeled Images](https://mlsanity.com/paper/2008.11586/) (2020)
- [Bag of Tricks and A Strong baseline for Image Copy Detection](https://mlsanity.com/paper/2111.08004/) (2021)
- [D2LV: A Data-Driven and Local-Verification Approach for Image Copy Detection](https://mlsanity.com/paper/2111.07090/) (2021)
- [HS-ResNet: Hierarchical-Split Block on Convolutional Neural Network](https://mlsanity.com/paper/2010.07621/) (2020)
- [eProduct: A Million-Scale Visual Search Benchmark to Address Product Recognition Challenges](https://mlsanity.com/paper/2107.05856/) (2021)

## Built on by (most cited first, 3 of 3)

- [DeCap: Decoding CLIP Latents for Zero-Shot Captioning via Text-Only Training](https://mlsanity.com/paper/2303.03032/) (2023)
- [Democratizing Fine-grained Visual Recognition with Large Language Models](https://mlsanity.com/paper/2401.13837/) (2024)
- [Vocabulary-free Fine-grained Visual Recognition via Enriched Contextually Grounded Vision-Language Model](https://mlsanity.com/paper/2507.23070/) (2025)

## Abstract (arXiv)

Product retrieval is of great importance in the ecommerce domain. This paper introduces our 1st-place solution in eBay eProduct Visual Search Challenge (FGVC9), which is featured for an ensemble of about 20 models from vision models and vision-language models. While model ensemble is common, we show that combining the vision models and vision-language models brings particular benefits from their complementarity and is a key factor to our superiority. Specifically, for the vision models, we use a two-stage training pipeline which first learns from the coarse labels provided in the training set and then conducts fine-grained self-supervised training, yielding a coarse-to-fine metric learning manner. For the vision-language models, we use the textual description of the training image as the supervision signals for fine-tuning the image-encoder (feature extractor). With these designs, our solution achieves 0.7623 MAR@10, ranking the first place among all the competitors. The code is available at: hrefhttps://github.com/WangWenhao0716/V2LV2L.