# Enhancing E-Commerce Recommendation using Pre-Trained Language Model and Fine-Tuning

arXiv: https://arxiv.org/abs/2302.04443 · HTML: https://mlsanity.com/paper/2302.04443/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 36 references that are in the map.
- First posted on arXiv 9 Feb 2023.

## Builds on (most cited first, 21 of 21)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Deep contextualized word representations](https://mlsanity.com/paper/1802.05365/) (2018)
- [Don't Stop Pretraining: Adapt Language Models to Domains and Tasks](https://mlsanity.com/paper/2004.10964/) (2020)
- [SpanBERT: Improving Pre-training by Representing and Predicting Spans](https://mlsanity.com/paper/1907.10529/) (2019)
- [Wide & Deep Learning for Recommender Systems](https://mlsanity.com/paper/1606.07792/) (2016)
- [Supervised Learning of Universal Sentence Representations from Natural Language Inference Data](https://mlsanity.com/paper/1705.02364/) (2017)
- [ERNIE: Enhanced Representation through Knowledge Integration](https://mlsanity.com/paper/1904.09223/) (2019)
- [The Natural Language Decathlon: Multitask Learning as Question Answering](https://mlsanity.com/paper/1806.08730/) (2018)
- [Understanding Random Forests: From Theory to Practice](https://mlsanity.com/paper/1407.7502/) (2014)
- [LadaBERT: Lightweight Adaptation of BERT through Hybrid Model Compression](https://mlsanity.com/paper/2004.04124/) (2020)
- [E-BERT: A Phrase and Product Knowledge Enhanced Language Model for E-commerce](https://mlsanity.com/paper/2009.02835/) (2020)
- [Language Modeling Teaches You More Syntax than Translation Does: Lessons Learned Through Auxiliary Task Analysis](https://mlsanity.com/paper/1809.10040/) (2018)
- [Fast and Simple PCA via Convex Optimization](https://mlsanity.com/paper/1509.05647/) (2015)
- [Regularizing Matrix Factorization with User and Item Embeddings for Recommendation](https://mlsanity.com/paper/1809.00979/) (2018)
- [Personalizing Similar Product Recommendations in Fashion E-commerce](https://mlsanity.com/paper/1806.11371/) (2018)
- [Personalized Advertisement Recommendation: A Ranking Approach to Address the Ubiquitous Click Sparsity Problem](https://mlsanity.com/paper/1603.01870/) (2016)
- [Utilizing Human Memory Processes to Model Genre Preferences for Personalized Music Recommendations](https://mlsanity.com/paper/2003.10699/) (2020)

## Built on by (most cited first, 1 of 1)

- [Recommendation systems in e-commerce applications with machine learning methods](https://mlsanity.com/paper/2506.17287/) (2025)

## Abstract (arXiv)

Pretrained Language Models (PLM) have been greatly successful on a board range of natural language processing (NLP) tasks. However, it has just started being applied to the domain of recommendation systems. Traditional recommendation algorithms failed to incorporate the rich textual information in e-commerce datasets, which hinderss the performance of those models. We present a thorough investigation on the effect of various strategy of incorporating PLMs into traditional recommender algorithms on one of the e-commerce datasets, and we compare the results with vanilla recommender baseline models. We show that the application of PLMs and domain specific fine-tuning lead to an increase on the predictive capability of combined models. These results accentuate the importance of utilizing textual information in the context of e-commerce, and provides insight on how to better apply PLMs alongside traditional recommender system algorithms. The code used in this paper is available on Github: https://github.com/NuofanXu/bert_retail_recommender.