# Intent-based Product Collections for E-commerce using Pretrained Language Models

arXiv: https://arxiv.org/abs/2110.08241 · HTML: https://mlsanity.com/paper/2110.08241/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 42 references that are in the map.
- First posted on arXiv 15 Oct 2021.

## Builds on (most cited first, 28 of 28)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Billion-scale similarity search with GPUs](https://mlsanity.com/paper/1702.08734/) (2017)
- [Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval](https://mlsanity.com/paper/2007.00808/) (2020)
- [ERNIE 2.0: A Continual Pre-training Framework for Language Understanding](https://mlsanity.com/paper/1907.12412/) (2019)
- [BERT for Joint Intent Classification and Slot Filling](https://mlsanity.com/paper/1902.10909/) (2019)
- [TOD-BERT: Pre-trained Natural Language Understanding for Task-Oriented Dialogue](https://mlsanity.com/paper/2004.06871/) (2020)
- [A Deep Relevance Matching Model for Ad-hoc Retrieval](https://mlsanity.com/paper/1711.08611/) (2017)
- [Top2Vec: Distributed Representations of Topics](https://mlsanity.com/paper/2008.09470/) (2020)
- [Convolutional Neural Network Architectures for Matching Natural Language Sentences](https://mlsanity.com/paper/1503.03244/) (2015)
- [What Changes Can Large-scale Language Models Bring? Intensive Study on HyperCLOVA: Billions-scale Korean Generative Pretrained Transformers](https://mlsanity.com/paper/2109.04650/) (2021)
- [Learning to Match Using Local and Distributed Representations of Text for Web Search](https://mlsanity.com/paper/1610.08136/) (2016)
- [Deep Sentence Embedding Using Long Short-Term Memory Networks: Analysis and Application to Information Retrieval](https://mlsanity.com/paper/1502.06922/) (2015)
- [Semantic Product Search](https://mlsanity.com/paper/1907.00937/) (2019)
- [Learning Heterogeneous Knowledge Base Embeddings for Explainable Recommendation](https://mlsanity.com/paper/1805.03352/) (2018)
- [AutoKnow: Self-Driving Knowledge Collection for Products of Thousands of Types](https://mlsanity.com/paper/2006.13473/) (2020)
- [Pre-trained Language Model for Web-scale Retrieval in Baidu Search](https://mlsanity.com/paper/2106.03373/) (2021)
- [AliCoCo: Alibaba E-commerce Cognitive Concept Net](https://mlsanity.com/paper/2003.13230/) (2020)
- [PACRR: A Position-Aware Neural IR Model for Relevance Matching](https://mlsanity.com/paper/1704.03940/) (2017)
- [One4all User Representation for Recommender Systems in E-commerce](https://mlsanity.com/paper/2106.00573/) (2021)
- [BERT Goes Shopping: Comparing Distributional Models for Product Representations](https://mlsanity.com/paper/2012.09807/) (2020)
- [A Dual Embedding Space Model for Document Ranking](https://mlsanity.com/paper/1602.01137/) (2016)
- [Explainable Entity-based Recommendations with Knowledge Graphs](https://mlsanity.com/paper/1707.05254/) (2017)
- [Using Large Pretrained Language Models for Answering User Queries from Product Specifications](https://mlsanity.com/paper/2005.14613/) (2020)
- [Co-PACRR: A Context-Aware Neural IR Model for Ad-hoc Retrieval](https://mlsanity.com/paper/1706.10192/) (2017)
- [Global-Local Item Embedding for Temporal Set Prediction](https://mlsanity.com/paper/2109.02074/) (2021)

## Built on by (most cited first, 1 of 1)

- [Recommender Systems in the Era of Large Language Models (LLMs)](https://mlsanity.com/paper/2307.02046/) (2023)

## Abstract (arXiv)

Building a shopping product collection has been primarily a human job. With the manual efforts of craftsmanship, experts collect related but diverse products with common shopping intent that are effective when displayed together, e.g., backpacks, laptop bags, and messenger bags for freshman bag gifts. Automatically constructing a collection requires an ML system to learn a complex relationship between the customer's intent and the product's attributes. However, there have been challenging points, such as 1) long and complicated intent sentences, 2) rich and diverse product attributes, and 3) a huge semantic gap between them, making the problem difficult. In this paper, we use a pretrained language model (PLM) that leverages textual attributes of web-scale products to make intent-based product collections. Specifically, we train a BERT with triplet loss by setting an intent sentence to an anchor and corresponding products to positive examples. Also, we improve the performance of the model by search-based negative sampling and category-wise positive pair augmentation. Our model significantly outperforms the search-based baseline model for intent-based product matching in offline evaluations. Furthermore, online experimental results on our e-commerce platform show that the PLM-based method can construct collections of products with increased CTR, CVR, and order-diversity compared to expert-crafted collections.