mlsanityData to 14 Oct 2026

ItemSage: Learning Product Embeddings for Shopping Recommendations at Pinterest

arXiv 2205.11728 ↗First posted 24 May 2022cs.IR cs.LGcited by 23

Not on the map. Why?

Abstract (arXiv)

Learned embeddings for products are an important building block for web-scale e-commerce recommendation systems. At Pinterest, we build a single set of product embeddings called ItemSage to provide relevant recommendations in all shopping use cases including user, image and search based recommendations. This approach has led to significant improvements in engagement and conversion metrics, while reducing both infrastructure and maintenance cost. While most prior work focuses on building product embeddings from features coming from a single modality, we introduce a transformer-based architecture capable of aggregating information from both text and image modalities and show that it significantly outperforms single modality baselines. We also utilize multi-task learning to make ItemSage optimized for several engagement types, leading to a candidate generation system that is efficient for all of the engagement objectives of the end-to-end recommendation system. Extensive offline experiments are conducted to illustrate the effectiveness of our approach and results from online A/B experiments show substantial gains in key business metrics (up to +7% gross merchandise value/user and +11% click volume).

Key facts. Cited by 23 papers in the map (data through 14 Oct 2026). Builds on 23 of its 39 references that are in the map. First posted on arXiv 24 May 2022.

Builds on

23 of 39 references.

References · 23

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2017Inductive Representation Learning on Large Graphscited by 6,320
  3. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,307
  4. 2019LXMERT: Learning Cross-Modality Encoder Representations from Transformerscited by 1,553
  5. 2019VisualBERT: A Simple and Performant Baseline for Vision and Languagecited by 1,296
  6. 2018Graph Convolutional Neural Networks for Web-Scale Recommender Systemscited by 1,138
  7. 2016Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphscited by 861
  8. 2017An Overview of Multi-Task Learning in Deep Neural Networkscited by 803
  9. 2020Heterogeneous Graph Transformercited by 582
  10. 2019Unified Vision-Language Pre-Training for Image Captioning and VQAcited by 494
  11. 2019UNITER: UNiversal Image-TExt Representation Learningcited by 228
  12. 2020Embedding-based Retrieval in Facebook Searchcited by 156
11 more
  1. 2020PinnerSage: Multi-Modal User Embedding Framework for Recommendations at Pinterestcited by 75
  2. 2017Hash Embeddings for Efficient Word Representationscited by 55
  3. 2019Semantic Product Searchcited by 51
  4. 2017Pixie: A System for Recommending 3+ Billion Items to 200+ Million Users in Real-Timecited by 47
  5. 2020Towards Personalized and Semantic Retrieval: An End-to-End Solution for E-commerce Search via Embedding Learningcited by 32
  6. 2019Learning a Unified Embedding for Visual Search at Pinterestcited by 25
  7. 2021Billion-Scale Pretraining with Vision Transformers for Multi-Task Visual Representationscited by 24
  8. 2017Modeling Multimodal Clues in a Hybrid Deep Learning Framework for Video Classificationcited by 13
  9. 2020DeText: A Deep Text Ranking Framework with BERTcited by 13
  10. 2021Graph-based Multilingual Product Retrieval in E-commerce Searchcited by 9
  11. 2021Lessons Learned Addressing Dataset Bias in Model-Based Candidate Generation at Twittercited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 5; 2024: 5; 2025: 9; 2026: 4.

23 papers.

Sort:

By area:

  1. 2024HLLM: Enhancing Sequential Recommendations via Hierarchical Large Language Models for Item and User Modelingcited by 63
  2. 2023Scaling User Modeling: Large-scale Online User Representations for Ads Personalization in Metacited by 21
  3. 2024OmniSearchSage: Multi-Task Multi-Entity Embeddings for Pinterest Searchcited by 13
  4. 2024Async Learned User Embeddings for Ads Delivery Optimizationcited by 7
  5. 2023adSformers: Personalization from Short-Term Sequences and Diversity of Representations in Etsy Adscited by 4
  6. 2025OmniSage: Large Scale, Multi-Entity Heterogeneous Graph Representation Learningcited by 4
  7. 2025Realizing Scaling Laws in Recommender Systems: A Foundation-Expert Paradigm for Hyperscale Model Deploymentcited by 4
  8. 2025Diffusion-based Multi-modal Synergy Interest Network for Click-through Rate Predictioncited by 4
  9. 2025DashCLIP: Leveraging multimodal models for generating semantic embeddings for DoorDashcited by 2
  10. 2023Efficient Large-Scale Visual Representation Learning And Evaluationcited by 1
  11. 2026Harmonized Tabular-Image Fusion via Gradient-Aligned Alternating Learningcited by 1
  12. 2026PinEqualizer: Full Funnel Content Exploration and Debiasing System at Pinterestcited by 1
11 more
  1. 2023NFTs to MARS: Multi-Attention Recommender System for NFTscited by 0
  2. 2023Natural language to SQL in low-code platformscited by 0
  3. 2024Pfeed: Generating near real-time personalized feeds using precomputed embedding similaritiescited by 0
  4. 2024PPM : A Pre-trained Plug-in Model for Click-through Rate Predictioncited by 0
  5. 2025Visual Product Graph: Bridging Visual Products And Composite Images For End-to-End Style Recommendationscited by 0
  6. 2025Cross-Domain Web Information Extraction at Pinterestcited by 0
  7. 2025Entity Representation Learning Through Onsite-Offsite Graph for Pinterest Adscited by 0
  8. 2025Large Foundation Model for Ads Recommendationcited by 0
  9. 2025Decoupled Entity Representation Learning for Pinterest Ads Rankingcited by 0
  10. 2026Deep-learning Causal Retrieval Optimization for Efficient e-commerce Distribution in Pinterestcited by 0
  11. 2026Pailitao-MMSearch: Building Native E-Commerce Multimodal Search Foundationcited by 0