mlsanityData to 14 Oct 2026

Improving Visual Recommendation on E-commerce Platforms Using Vision-Language Models

arXiv 2510.13359 ↗First posted 15 Oct 2025cs.IR cs.CV cs.LGcited by 2

On the map: Multimodal LLMs / vision-language models → Document understanding and multimodal RAG

Abstract (arXiv)

On large-scale e-commerce platforms with tens of millions of active monthly users, recommending visually similar products is essential for enabling users to efficiently discover items that align with their preferences. This study presents the application of a vision-language model (VLM) -- which has demonstrated strong performance in image recognition and image-text retrieval tasks -- to product recommendations on Mercari, a major consumer-to-consumer marketplace used by more than 20 million monthly users in Japan. Specifically, we fine-tuned SigLIP, a VLM employing a sigmoid-based contrastive loss, using one million product image-title pairs from Mercari collected over a three-month period, and developed an image encoder for generating item embeddings used in the recommendation system. Our evaluation comprised an offline analysis of historical interaction logs and an online A/B test in a production environment. In offline analysis, the model achieved a 9.1% improvement in nDCG@5 compared with the baseline. In the online A/B test, the click-through rate improved by 50% whereas the conversion rate improved by 14% compared with the existing model. These results demonstrate the effectiveness of VLM-based encoders for e-commerce product recommendations and provide practical insights into the development of visual similarity-based recommendation systems.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Document understanding and multimodal RAG, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 11 of its 15 references that are in the map. First posted on arXiv 15 Oct 2025.

Builds on

11 of 15 references.

References · 11

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,795
  4. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,845
  5. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,109
  6. 2015Flickr30k Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Modelscited by 1,362
  7. 2022PaLI: A Jointly-Scaled Multilingual Language-Image Modelcited by 766
  8. 2021CLIP2Video: Mastering Video-Text Retrieval via Image CLIPcited by 225
  9. 2022CLIP-Art: Contrastive Pre-training for Fine-Grained Art Classificationcited by 65
  10. 2017Visual Search at eBaycited by 25
  11. 2019Learning a Unified Embedding for Visual Search at Pinterestcited by 25

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026LookBench: A Live and Holistic Open Benchmark for Fashion Image Retrievalcited by 7
  2. 2026Through Their Eyes: Fixation-aligned Tuning for Personalized User Emulationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025VL-CLIP: Enhancing Multimodal Recommendations via Visual Grounding and LLM-Augmented CLIP Embeddings
  2. 2025Embedding the Teacher: Distilling vLLM Preferences for Scalable Image Retrieval
  3. 2024Image Score: Learning and Evaluating Human Preferences for Mercari Search
  4. 2026PixRec: Leveraging Visual Context for Next-Item Prediction in Sequential Recommendation
  5. 2026Adapting Vision-Language Models for E-commerce Understanding at Scale
  6. 2025Zero-Shot Retrieval for Scalable Visual Search in a Two-Sided Marketplace
  7. 2026Text-Guided Visual Representation Learning for Robust Multimodal E-Commerce Recommendation ↗
  8. 2022V2L: Leveraging Vision and Vision-language Models into Large-scale Product Retrieval