# Visual Search at eBay

arXiv: https://arxiv.org/abs/1706.03154 · HTML: https://mlsanity.com/paper/1706.03154/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Jun 2017.

## Built on by (most cited first, 25 of 25)

- [Survey: Leakage and Privacy at Inference Time](https://mlsanity.com/paper/2107.01614/) (2021)
- [Visual Search at Alibaba](https://mlsanity.com/paper/2102.04674/) (2021)
- [Billion-Scale Pretraining with Vision Transformers for Multi-Task Visual Representations](https://mlsanity.com/paper/2108.05887/) (2021)
- [Bringing Multimodality to Amazon Visual Search System](https://mlsanity.com/paper/2412.13364/) (2024)
- [Towards Zero-shot Cross-lingual Image Retrieval and Tagging](https://mlsanity.com/paper/2109.07622/) (2021)
- [Smart Fashion: A Review of AI Applications in the Fashion & Apparel Industry](https://mlsanity.com/paper/2111.00905/) (2021)
- [LRVS-Fashion: Extending Visual Search with Referring Instructions](https://mlsanity.com/paper/2306.02928/) (2023)
- [Qilin: A Multimodal Information Retrieval Dataset with APP-level User Sessions](https://mlsanity.com/paper/2503.00501/) (2025)
- [MMFL-Net: Multi-scale and Multi-granularity Feature Learning for Cross-domain Fashion Retrieval](https://mlsanity.com/paper/2210.15128/) (2022)
- [OneVision: An End-to-End Generative Framework for Multi-view E-commerce Vision Search](https://mlsanity.com/paper/2510.05759/) (2025)
- [Improving Visual Recommendation on E-commerce Platforms Using Vision-Language Models](https://mlsanity.com/paper/2510.13359/) (2025)
- [Visually Similar Products Retrieval for Shopsy](https://mlsanity.com/paper/2210.04560/) (2022)
- [Efficient Large-Scale Visual Representation Learning And Evaluation](https://mlsanity.com/paper/2305.13399/) (2023)
- [Learning with Multi-modal Gradient Attention for Explainable Composed Image Retrieval](https://mlsanity.com/paper/2308.16649/) (2023)
- [FashionFail: Addressing Failure Cases in Fashion Object Detection and Segmentation](https://mlsanity.com/paper/2404.08582/) (2024)
- [Virtual ID Discovery from E-commerce Media at Alibaba: Exploiting Richness of User Click Behavior for Visual Search Relevance](https://mlsanity.com/paper/2102.04667/) (2021)
- [Web-Scale Generic Object Detection at Microsoft Bing](https://mlsanity.com/paper/2107.01814/) (2021)
- [Technologies for AI-Driven Fashion Social Networking Service with E-Commerce](https://mlsanity.com/paper/2203.10996/) (2022)
- [Adaptive Cross Batch Normalization for Metric Learning](https://mlsanity.com/paper/2303.17127/) (2023)
- [ITEm: Unsupervised Image-Text Embedding Learning for eCommerce](https://mlsanity.com/paper/2311.02084/) (2023)
- [Zero-Shot Retrieval for Scalable Visual Search in a Two-Sided Marketplace](https://mlsanity.com/paper/2508.05661/) (2025)
- [From Pixels to Purchase: Building and Evaluating a Taxonomy-Decoupled Visual Search Engine for Home Goods E-commerce](https://mlsanity.com/paper/2601.11769/) (2026)
- [Pailitao-MMSearch: Building Native E-Commerce Multimodal Search Foundation](https://mlsanity.com/paper/2607.17499/) (2026)
- [RegionFed: Federated Learning for Personalized Query Understanding in Heterogeneous Retail Environments](https://mlsanity.com/paper/2609.05403/) (2026)
- [Graded-Relevance Composed Multimodal Retrieval for E-commerce Visual Search at Scale](https://mlsanity.com/paper/2609.24152/) (2026)

## Abstract (arXiv)

In this paper, we propose a novel end-to-end approach for scalable visual search infrastructure. We discuss the challenges we faced for a massive volatile inventory like at eBay and present our solution to overcome those. We harness the availability of large image collection of eBay listings and state-of-the-art deep learning techniques to perform visual search at scale. Supervised approach for optimized search limited to top predicted categories and also for compact binary signature are key to scale up without compromising accuracy and precision. Both use a common deep neural network requiring only a single forward inference. The system architecture is presented with in-depth discussions of its basic components and optimizations for a trade-off between search relevance and latency. This solution is currently deployed in a distributed cloud infrastructure and fuels visual search in eBay ShopBot and Close5. We show benchmark on ImageNet dataset on which our approach is faster and more accurate than several unsupervised baselines. We share our learnings with the hope that visual search becomes a first class citizen for all large scale search engines rather than an afterthought.