# OmniSearchSage: Multi-Task Multi-Entity Embeddings for Pinterest Search

arXiv: https://arxiv.org/abs/2404.16260 · HTML: https://mlsanity.com/paper/2404.16260/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 40 references that are in the map.
- First posted on arXiv 25 Apr 2024.

## Builds on (most cited first, 20 of 20)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation](https://mlsanity.com/paper/2201.12086/) (2022)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [Billion-scale similarity search with GPUs](https://mlsanity.com/paper/1702.08734/) (2017)
- [Enriching Word Vectors with Subword Information](https://mlsanity.com/paper/1607.04606/) (2016)
- [Graph Convolutional Neural Networks for Web-Scale Recommender Systems](https://mlsanity.com/paper/1806.01973/) (2018)
- [Latent Retrieval for Weakly Supervised Open Domain Question Answering](https://mlsanity.com/paper/1906.00300/) (2019)
- [Supervised Learning of Universal Sentence Representations from Natural Language Inference Data](https://mlsanity.com/paper/1705.02364/) (2017)
- [DeCLUTR: Deep Contrastive Learning for Unsupervised Textual Representations](https://mlsanity.com/paper/2006.03659/) (2020)
- [Document Expansion by Query Prediction](https://mlsanity.com/paper/1904.08375/) (2019)
- [Embedding-based Retrieval in Facebook Search](https://mlsanity.com/paper/2006.11632/) (2020)
- [Hash Embeddings for Efficient Word Representations](https://mlsanity.com/paper/1709.03933/) (2017)
- [Semantic Product Search](https://mlsanity.com/paper/1907.00937/) (2019)
- [Embedding-based Product Retrieval in Taobao Search](https://mlsanity.com/paper/2106.09297/) (2021)
- [CommerceMM: Large-Scale Commerce MultiModal Representation Learning with Omni Retrieval](https://mlsanity.com/paper/2202.07247/) (2022)
- [Billion-Scale Pretraining with Vision Transformers for Multi-Task Visual Representations](https://mlsanity.com/paper/2108.05887/) (2021)
- [ItemSage: Learning Product Embeddings for Shopping Recommendations at Pinterest](https://mlsanity.com/paper/2205.11728/) (2022)
- [Uni-Retriever: Towards Learning The Unified Embedding Based Retriever in Bing Sponsored Search](https://mlsanity.com/paper/2202.06212/) (2022)

## Built on by (most cited first, 13 of 13)

- [Improving Pinterest Search Relevance Using Large Language Models](https://mlsanity.com/paper/2410.17152/) (2024)
- [PinRec: Unified Generative Retrieval for Pinterest Recommender Systems](https://mlsanity.com/paper/2504.10507/) (2025)
- [OmniSage: Large Scale, Multi-Entity Heterogeneous Graph Representation Learning](https://mlsanity.com/paper/2504.17811/) (2025)
- [InteractRank: Personalized Web-Scale Search Pre-Ranking with Cross Interaction Features](https://mlsanity.com/paper/2504.06609/) (2025)
- [Warmer for Less: A Cost-Efficient Strategy for Cold-Start Recommendations at Pinterest](https://mlsanity.com/paper/2512.17277/) (2025)
- [Multi-Faceted Large Embedding Tables for Pinterest Ads Ranking](https://mlsanity.com/paper/2508.05700/) (2025)
- [Applying Embedding-Based Retrieval to Airbnb Search](https://mlsanity.com/paper/2601.06873/) (2026)
- [UniPinRec: Unifying Generative Retrieval and Ranking at Pinterest Scale](https://mlsanity.com/paper/2606.00422/) (2026)
- [Long-term User Engagement Optimization through Model-agnostic Downstream Rewards Learning](https://mlsanity.com/paper/2607.14192/) (2026)
- [Cross-Domain Web Information Extraction at Pinterest](https://mlsanity.com/paper/2508.01096/) (2025)
- [Decoupled Entity Representation Learning for Pinterest Ads Ranking](https://mlsanity.com/paper/2509.04337/) (2025)
- [From Pixels to Purchase: Building and Evaluating a Taxonomy-Decoupled Visual Search Engine for Home Goods E-commerce](https://mlsanity.com/paper/2601.11769/) (2026)
- [PinCLIP: Large-scale Foundational Multimodal Representation at Pinterest](https://mlsanity.com/paper/2603.03544/) (2026)

## Abstract (arXiv)

In this paper, we present OmniSearchSage, a versatile and scalable system for understanding search queries, pins, and products for Pinterest search. We jointly learn a unified query embedding coupled with pin and product embeddings, leading to an improvement of >8% relevance, >7% engagement, and >5% ads CTR in Pinterest's production search system. The main contributors to these gains are improved content understanding, better multi-task learning, and real-time serving. We enrich our entity representations using diverse text derived from image captions from a generative LLM, historical engagement, and user-curated boards. Our multitask learning setup produces a single search query embedding in the same space as pin and product embeddings and compatible with pre-existing pin and product embeddings. We show the value of each feature through ablation studies, and show the effectiveness of a unified model compared to standalone counterparts. Finally, we share how these embeddings have been deployed across the Pinterest search stack, from retrieval to ranking, scaling to serve 300k requests per second at low latency. Our implementation of this work is available at https://github.com/pinterest/atg-research/tree/main/omnisearchsage.