# BERT Goes Shopping: Comparing Distributional Models for Product Representations

arXiv: https://arxiv.org/abs/2012.09807 · HTML: https://mlsanity.com/paper/2012.09807/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Dec 2020.

## Built on by (most cited first, 7 of 7)

- [Contrastive Language-Image Pre-training for the Italian Language](https://mlsanity.com/paper/2108.08688/) (2021)
- [Query2Prod2Vec Grounded Word Embeddings for eCommerce](https://mlsanity.com/paper/2104.02061/) (2021)
- [Beyond NDCG: behavioral testing of recommender systems with RecList](https://mlsanity.com/paper/2111.09963/) (2021)
- [Transformers with multi-modal features and post-fusion context for e-commerce session-based recommendation](https://mlsanity.com/paper/2107.05124/) (2021)
- [Reasonable Scale Machine Learning with Open-Source Metaflow](https://mlsanity.com/paper/2303.11761/) (2023)
- ["Does it come in black?" CLIP-like models are zero-shot recommenders](https://mlsanity.com/paper/2204.02473/) (2022)
- [Intent-based Product Collections for E-commerce using Pretrained Language Models](https://mlsanity.com/paper/2110.08241/) (2021)

## Abstract (arXiv)

Word embeddings (e.g., word2vec) have been applied successfully to eCommerce products through~textitprod2vec. Inspired by the recent performance improvements on several NLP tasks brought by contextualized embeddings, we propose to transfer BERT-like architectures to eCommerce: our model -- ~textitProd2BERT -- is trained to generate representations of products through masked session modeling. Through extensive experiments over multiple shops, different tasks, and a range of design choices, we systematically compare the accuracy of~textitProd2BERT and~textitprod2vec embeddings: while~textitProd2BERT is found to be superior in several scenarios, we highlight the importance of resources and hyperparameters in the best performing models. Finally, we provide guidelines to practitioners for training embeddings under a variety of computational and data constraints.