# ItemSage: Learning Product Embeddings for Shopping Recommendations at Pinterest

arXiv: https://arxiv.org/abs/2205.11728 · HTML: https://mlsanity.com/paper/2205.11728/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 39 references that are in the map.
- First posted on arXiv 24 May 2022.

## Builds on (most cited first, 23 of 23)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [LXMERT: Learning Cross-Modality Encoder Representations from Transformers](https://mlsanity.com/paper/1908.07490/) (2019)
- [VisualBERT: A Simple and Performant Baseline for Vision and Language](https://mlsanity.com/paper/1908.03557/) (2019)
- [Graph Convolutional Neural Networks for Web-Scale Recommender Systems](https://mlsanity.com/paper/1806.01973/) (2018)
- [Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphs](https://mlsanity.com/paper/1603.09320/) (2016)
- [An Overview of Multi-Task Learning in Deep Neural Networks](https://mlsanity.com/paper/1706.05098/) (2017)
- [Heterogeneous Graph Transformer](https://mlsanity.com/paper/2003.01332/) (2020)
- [Unified Vision-Language Pre-Training for Image Captioning and VQA](https://mlsanity.com/paper/1909.11059/) (2019)
- [UNITER: UNiversal Image-TExt Representation Learning](https://mlsanity.com/paper/1909.11740/) (2019)
- [Embedding-based Retrieval in Facebook Search](https://mlsanity.com/paper/2006.11632/) (2020)
- [PinnerSage: Multi-Modal User Embedding Framework for Recommendations at Pinterest](https://mlsanity.com/paper/2007.03634/) (2020)
- [Hash Embeddings for Efficient Word Representations](https://mlsanity.com/paper/1709.03933/) (2017)
- [Semantic Product Search](https://mlsanity.com/paper/1907.00937/) (2019)
- [Pixie: A System for Recommending 3+ Billion Items to 200+ Million Users in Real-Time](https://mlsanity.com/paper/1711.07601/) (2017)
- [Towards Personalized and Semantic Retrieval: An End-to-End Solution for E-commerce Search via Embedding Learning](https://mlsanity.com/paper/2006.02282/) (2020)
- [Learning a Unified Embedding for Visual Search at Pinterest](https://mlsanity.com/paper/1908.01707/) (2019)
- [Billion-Scale Pretraining with Vision Transformers for Multi-Task Visual Representations](https://mlsanity.com/paper/2108.05887/) (2021)
- [Modeling Multimodal Clues in a Hybrid Deep Learning Framework for Video Classification](https://mlsanity.com/paper/1706.04508/) (2017)
- [DeText: A Deep Text Ranking Framework with BERT](https://mlsanity.com/paper/2008.02460/) (2020)
- [Graph-based Multilingual Product Retrieval in E-commerce Search](https://mlsanity.com/paper/2105.02978/) (2021)
- [Lessons Learned Addressing Dataset Bias in Model-Based Candidate Generation at Twitter](https://mlsanity.com/paper/2105.09293/) (2021)

## Built on by (most cited first, 23 of 23)

- [HLLM: Enhancing Sequential Recommendations via Hierarchical Large Language Models for Item and User Modeling](https://mlsanity.com/paper/2409.12740/) (2024)
- [Scaling User Modeling: Large-scale Online User Representations for Ads Personalization in Meta](https://mlsanity.com/paper/2311.09544/) (2023)
- [OmniSearchSage: Multi-Task Multi-Entity Embeddings for Pinterest Search](https://mlsanity.com/paper/2404.16260/) (2024)
- [Async Learned User Embeddings for Ads Delivery Optimization](https://mlsanity.com/paper/2406.05898/) (2024)
- [adSformers: Personalization from Short-Term Sequences and Diversity of Representations in Etsy Ads](https://mlsanity.com/paper/2302.01255/) (2023)
- [OmniSage: Large Scale, Multi-Entity Heterogeneous Graph Representation Learning](https://mlsanity.com/paper/2504.17811/) (2025)
- [Realizing Scaling Laws in Recommender Systems: A Foundation-Expert Paradigm for Hyperscale Model Deployment](https://mlsanity.com/paper/2508.02929/) (2025)
- [Diffusion-based Multi-modal Synergy Interest Network for Click-through Rate Prediction](https://mlsanity.com/paper/2508.21460/) (2025)
- [DashCLIP: Leveraging multimodal models for generating semantic embeddings for DoorDash](https://mlsanity.com/paper/2504.07110/) (2025)
- [Efficient Large-Scale Visual Representation Learning And Evaluation](https://mlsanity.com/paper/2305.13399/) (2023)
- [Harmonized Tabular-Image Fusion via Gradient-Aligned Alternating Learning](https://mlsanity.com/paper/2604.01579/) (2026)
- [PinEqualizer: Full Funnel Content Exploration and Debiasing System at Pinterest](https://mlsanity.com/paper/2607.22518/) (2026)
- [NFTs to MARS: Multi-Attention Recommender System for NFTs](https://mlsanity.com/paper/2306.10053/) (2023)
- [Natural language to SQL in low-code platforms](https://mlsanity.com/paper/2308.15239/) (2023)
- [Pfeed: Generating near real-time personalized feeds using precomputed embedding similarities](https://mlsanity.com/paper/2402.16073/) (2024)
- [PPM : A Pre-trained Plug-in Model for Click-through Rate Prediction](https://mlsanity.com/paper/2403.10049/) (2024)
- [Visual Product Graph: Bridging Visual Products And Composite Images For End-to-End Style Recommendations](https://mlsanity.com/paper/2505.21454/) (2025)
- [Cross-Domain Web Information Extraction at Pinterest](https://mlsanity.com/paper/2508.01096/) (2025)
- [Entity Representation Learning Through Onsite-Offsite Graph for Pinterest Ads](https://mlsanity.com/paper/2508.02609/) (2025)
- [Large Foundation Model for Ads Recommendation](https://mlsanity.com/paper/2508.14948/) (2025)
- [Decoupled Entity Representation Learning for Pinterest Ads Ranking](https://mlsanity.com/paper/2509.04337/) (2025)
- [Deep-learning Causal Retrieval Optimization for Efficient e-commerce Distribution in Pinterest](https://mlsanity.com/paper/2607.14161/) (2026)
- [Pailitao-MMSearch: Building Native E-Commerce Multimodal Search Foundation](https://mlsanity.com/paper/2607.17499/) (2026)

## Abstract (arXiv)

Learned embeddings for products are an important building block for web-scale e-commerce recommendation systems. At Pinterest, we build a single set of product embeddings called ItemSage to provide relevant recommendations in all shopping use cases including user, image and search based recommendations. This approach has led to significant improvements in engagement and conversion metrics, while reducing both infrastructure and maintenance cost. While most prior work focuses on building product embeddings from features coming from a single modality, we introduce a transformer-based architecture capable of aggregating information from both text and image modalities and show that it significantly outperforms single modality baselines. We also utilize multi-task learning to make ItemSage optimized for several engagement types, leading to a candidate generation system that is efficient for all of the engagement objectives of the end-to-end recommendation system. Extensive offline experiments are conducted to illustrate the effectiveness of our approach and results from online A/B experiments show substantial gains in key business metrics (up to +7% gross merchandise value/user and +11% click volume).