# RecoBERT: A Catalog Language Model for Text-Based Recommendations

arXiv: https://arxiv.org/abs/2009.13292 · HTML: https://mlsanity.com/paper/2009.13292/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Sep 2020.

## Built on by (most cited first, 8 of 8)

- [Large Language Models are Competitive Near Cold-start Recommenders for Language- and Item-based Preferences](https://mlsanity.com/paper/2307.14225/) (2023)
- [Grad-SAM: Explaining Transformers via Gradient Self-Attention Maps](https://mlsanity.com/paper/2204.11073/) (2022)
- [Self-Supervised Document Similarity Ranking via Contextualized Language Models and Hierarchical Inference](https://mlsanity.com/paper/2106.01186/) (2021)
- [Interpreting BERT-based Text Similarity via Activation and Saliency Maps](https://mlsanity.com/paper/2208.06612/) (2022)
- [Cold Item Integration in Deep Hybrid Recommenders via Tunable Stochastic Gates](https://mlsanity.com/paper/2112.07615/) (2021)
- [CLAUSEREC: A Clause Recommendation Framework for AI-aided Contract Authoring](https://mlsanity.com/paper/2110.15794/) (2021)
- [Unintended Bias in Language Model-driven Conversational Recommendation](https://mlsanity.com/paper/2201.06224/) (2022)
- [MetricBERT: Text Representation Learning via Self-Supervised Triplet Training](https://mlsanity.com/paper/2208.06610/) (2022)

## Abstract (arXiv)

Language models that utilize extensive self-supervised pre-training from unlabeled text, have recently shown to significantly advance the state-of-the-art performance in a variety of language understanding tasks. However, it is yet unclear if and how these recent models can be harnessed for conducting text-based recommendations. In this work, we introduce RecoBERT, a BERT-based approach for learning catalog-specialized language models for text-based item recommendations. We suggest novel training and inference procedures for scoring similarities between pairs of items, that don't require item similarity labels. Both the training and the inference techniques were designed to utilize the unlabeled structure of textual catalogs, and minimize the discrepancy between them. By incorporating four scores during inference, RecoBERT can infer text-based item-to-item similarities more accurately than other techniques. In addition, we introduce a new language understanding task for wine recommendations using similarities based on professional wine reviews. As an additional contribution, we publish annotated recommendations dataset crafted by human wine experts. Finally, we evaluate RecoBERT and compare it to various state-of-the-art NLP models on wine and fashion recommendations tasks.