mlsanityData to 14 Oct 2026

e-CLIP: Large-Scale Vision-Language Representation Learning in E-commerce

arXiv 2207.00208 ↗First posted 1 Jul 2022cs.LG cs.CVcited by 10

Not on the map. Why?

Abstract (arXiv)

Understanding vision and language representations of product content is vital for search and recommendation applications in e-commerce. As a backbone for online shopping platforms and inspired by the recent success in representation learning research, we propose a contrastive learning framework that aligns language and visual models using unlabeled raw product text and images. We present techniques we used to train large-scale representation learning models and share solutions that address domain-specific challenges. We study the performance using our pre-trained model as backbones for diverse downstream tasks, including category classification, attribute extraction, product matching, product clustering, and adult product recognition. Experimental results show that our proposed method outperforms the baseline in each downstream task regarding both single modality and multiple modalities.

Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Builds on 48 of its 77 references that are in the map. First posted on arXiv 1 Jul 2022.

Builds on

48 of 77 references.

References · 48

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  4. 2020Language Models are Few-Shot Learnerscited by 32,419
  5. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  6. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  7. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  8. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  9. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  10. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  11. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  12. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
36 more
  1. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,864
  2. 2020Exploring Simple Siamese Representation Learningcited by 2,866
  3. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,857
  4. 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,222
  5. 2019LXMERT: Learning Cross-Modality Encoder Representations from Transformerscited by 1,553
  6. 2019VisualBERT: A Simple and Performant Baseline for Vision and Languagecited by 1,296
  7. 2020Oscar: Object-Semantics Aligned Pre-training for Vision-Language Taskscited by 1,214
  8. 2017Mixed Precision Trainingcited by 1,156
  9. 2019VideoBERT: A Joint Model for Video and Language Representation Learningcited by 703
  10. 2021Multimodal Few-Shot Learning with Frozen Language Modelscited by 661
  11. 2020Contrastive Learning with Hard Negative Samplescited by 628
  12. 2021LiT: Zero-Shot Transfer with Locked-image text Tuningcited by 577
  13. 2021SLIP: Self-supervision meets Language-Image Pre-trainingcited by 436
  14. 2021Unifying Vision-and-Language Tasks via Text Generationcited by 412
  15. 2021Supervision Exists Everywhere: A Data Efficient Contrastive Language-Image Pre-training Paradigmcited by 405
  16. 2020VirTex: Learning Visual Representations from Textual Annotationscited by 339
  17. 2017Don't Decay the Learning Rate, Increase the Batch Sizecited by 320
  18. 2021Combined Scaling for Zero-shot Transfer Learningcited by 182
  19. 2018Deep learning in agriculture: A surveycited by 160
  20. 2020Deep Entity Matching with Pre-Trained Language Modelscited by 144
  21. 2021Multimodal Contrastive Training for Visual Representation Learningcited by 124
  22. 2020Learning Visual Representations with Caption Annotationscited by 123
  23. 2018OpenTag: Open Attribute Value Extraction from Product Profiles [Deep Learning, Active Learning, Named Entity Recognition]cited by 63
  24. 2022DeepDPM: Deep Clustering With an Unknown Number of Clusterscited by 52
  25. 2021ViDT: An Efficient and Effective Fully Transformer-based Object Detectorcited by 46
  26. 2019Product Knowledge Graph Embedding for E-commercecited by 36
  27. 2020Towards Personalized and Semantic Retrieval: An End-to-End Solution for E-commerce Search via Embedding Learningcited by 32
  28. 2020AutoKnow: Self-Driving Knowledge Collection for Products of Thousands of Typescited by 31
  29. 2017Cascade Ranking for Operational E-commerce Searchcited by 29
  30. 2021Billion-Scale Pretraining with Vision Transformers for Multi-Task Visual Representationscited by 24
  31. 2022OA-Mine: Open-World Attribute Mining for E-Commerce Products with Weak Supervisioncited by 20
  32. 2020E-BERT: A Phrase and Product Knowledge Enhanced Language Model for E-commercecited by 19
  33. 2021Learning From Long-Tailed Data With Noisy Labelscited by 17
  34. 2020Shop The Look: Building a Large Scale Visual Shopping System at Pinterestcited by 12
  35. 2022Large-scale Bilingual Language-Image Contrastive Learningcited by 12
  36. 2021Large-Scale Training System for 100-Million Classification at Alibabacited by 10

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 2; 2024: 1; 2025: 5; 2026: 2.

10 papers.

  1. 2023RemoteCLIP: A Vision Language Foundation Model for Remote Sensingcited by 363
  2. 2025Adaptive Event Stream Slicing for Open-Vocabulary Event-Based Object Detection via Vision-Language Knowledge Distillationcited by 8
  3. 2024Semantic Shield: Defending Vision-Language Models Against Backdooring and Poisoning via Fine-grained Knowledge Alignmentcited by 7
  4. 2023VITR: Augmenting Vision Transformers with Relation-Focused Learning for Cross-Modal Information Retrievalcited by 1
  5. 2025DALIP: Distribution Alignment-based Language-Image Pre-Training for Domain-Specific Datacited by 1
  6. 2025Multimodal RAG Enhanced Visual Descriptioncited by 1
  7. 2025Multi-Modality Transformer for E-Commerce: Inferring User Purchase Intention to Bridge the Query-Product Gapcited by 0
  8. 2025Zero-Shot Retrieval for Scalable Visual Search in a Two-Sided Marketplacecited by 0
  9. 2026Geometry-Aware CLIP Retrieval via Local Cross-Modal Alignment and Steeringcited by 0
  10. 2026Rethinking Text-Based Image Retrieval in Specific Domaincited by 0