# Improving Pinterest Search Relevance Using Large Language Models

arXiv: https://arxiv.org/abs/2410.17152 · HTML: https://mlsanity.com/paper/2410.17152/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 28 references that are in the map.
- First posted on arXiv 22 Oct 2024.

## Builds on (most cited first, 21 of 21)

- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation](https://mlsanity.com/paper/2201.12086/) (2022)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [Dense Passage Retrieval for Open-Domain Question Answering](https://mlsanity.com/paper/2004.04906/) (2020)
- [QLoRA: Efficient Finetuning of Quantized LLMs](https://mlsanity.com/paper/2305.14314/) (2023)
- [MS MARCO: A Human Generated MAchine Reading COmprehension Dataset](https://mlsanity.com/paper/1611.09268/) (2016)
- [DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharing](https://mlsanity.com/paper/2111.09543/) (2021)
- [Graph Convolutional Neural Networks for Web-Scale Recommender Systems](https://mlsanity.com/paper/1806.01973/) (2018)
- [Training Deep Nets with Sublinear Memory Cost](https://mlsanity.com/paper/1604.06174/) (2016)
- [Revisiting Deep Learning Models for Tabular Data](https://mlsanity.com/paper/2106.11959/) (2021)
- [Passage Re-ranking with BERT](https://mlsanity.com/paper/1901.04085/) (2019)
- [Document Ranking with a Pretrained Sequence-to-Sequence Model](https://mlsanity.com/paper/2003.06713/) (2020)
- [Large Language Models for Information Retrieval: A Survey](https://mlsanity.com/paper/2308.07107/) (2023)
- [Distilling Task-Specific Knowledge from BERT into Simple Neural Networks](https://mlsanity.com/paper/1903.12136/) (2019)
- [Improving Efficient Neural Ranking Models with Cross-Architecture Knowledge Distillation](https://mlsanity.com/paper/2010.02666/) (2020)
- [RankT5: Fine-Tuning T5 for Text Ranking with Ranking Losses](https://mlsanity.com/paper/2210.10634/) (2022)
- [Shopping Queries Dataset: A Large-Scale ESCI Benchmark for Improving Product Search](https://mlsanity.com/paper/2206.06588/) (2022)
- [Learning a Unified Embedding for Visual Search at Pinterest](https://mlsanity.com/paper/1908.01707/) (2019)
- [Billion-Scale Pretraining with Vision Transformers for Multi-Task Visual Representations](https://mlsanity.com/paper/2108.05887/) (2021)
- [Understanding BERT Rankers Under Distillation](https://mlsanity.com/paper/2007.11088/) (2020)
- [OmniSearchSage: Multi-Task Multi-Entity Embeddings for Pinterest Search](https://mlsanity.com/paper/2404.16260/) (2024)

## Built on by (most cited first, 8 of 8)

- [From Reasoning LLMs to BERT: A Two-Stage Distillation Framework for Search Relevance](https://mlsanity.com/paper/2510.11056/) (2025)
- [To Judge or not to Judge: Using LLM Judgements for Advertiser Keyphrase Relevance at eBay](https://mlsanity.com/paper/2505.04209/) (2025)
- [Scaling Up Efficient Small Language Models Serving and Deployment for Semantic Job Search](https://mlsanity.com/paper/2510.22101/) (2025)
- [A Little More Like This: Text-to-Image Retrieval with Vision-Language Models Using Relevance Feedback](https://mlsanity.com/paper/2511.17255/) (2025)
- [LLMDistill4Ads: Using Cross-Encoders to Distill from LLM Signals for Advertiser Keyphrase Recommendations at eBay](https://mlsanity.com/paper/2508.03628/) (2025)
- [ADORE: Autonomous Domain-Oriented Relevance Engine for E-commerce](https://mlsanity.com/paper/2512.02555/) (2025)
- [K-CARE: Knowledge-driven Symmetrical Contextual Anchoring and Analogical Prototype Reasoning for E-commerce Relevance](https://mlsanity.com/paper/2604.25683/) (2026)
- [From Pixels to Purchase: Building and Evaluating a Taxonomy-Decoupled Visual Search Engine for Home Goods E-commerce](https://mlsanity.com/paper/2601.11769/) (2026)

## Abstract (arXiv)

To improve relevance scoring on Pinterest Search, we integrate Large Language Models (LLMs) into our search relevance model, leveraging carefully designed text representations to predict the relevance of Pins effectively. Our approach uses search queries alongside content representations that include captions extracted from a generative visual language model. These are further enriched with link-based text data, historically high-quality engaged queries, user-curated boards, Pin titles and Pin descriptions, creating robust models for predicting search relevance. We use a semi-supervised learning approach to efficiently scale up the amount of training data, expanding beyond the expensive human labeled data available. By utilizing multilingual LLMs, our system extends training data to include unseen languages and domains, despite initial data and annotator expertise being confined to English. Furthermore, we distill from the LLM-based model into real-time servable model architectures and features. We provide comprehensive offline experimental validation for our proposed techniques and demonstrate the gains achieved through the final deployed system at scale.