# A Transformer-Based Substitute Recommendation Model Incorporating Weakly Supervised Customer Behavior Data

arXiv: https://arxiv.org/abs/2211.02533 · HTML: https://mlsanity.com/paper/2211.02533/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 39 references that are in the map.
- First posted on arXiv 4 Nov 2022.

## Builds on (most cited first, 19 of 19)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [HuggingFace's Transformers: State-of-the-art Natural Language Processing](https://mlsanity.com/paper/1910.03771/) (2019)
- [Enriching Word Vectors with Subword Information](https://mlsanity.com/paper/1607.04606/) (2016)
- [Top-K Off-Policy Correction for a REINFORCE Recommender System](https://mlsanity.com/paper/1812.02353/) (2018)
- [Fast Matrix Factorization for Online Recommendation with Implicit Feedback](https://mlsanity.com/paper/1708.05024/) (2017)
- [Inferring Networks of Substitutable and Complementary Products](https://mlsanity.com/paper/1506.08839/) (2015)
- [Text Matching as Image Recognition](https://mlsanity.com/paper/1602.06359/) (2016)
- [Embedding-based Product Retrieval in Taobao Search](https://mlsanity.com/paper/2106.09297/) (2021)
- [Correcting for Selection Bias in Learning-to-rank Systems](https://mlsanity.com/paper/2001.11358/) (2020)
- [Try This Instead: Personalized and Interpretable Substitute Recommendation](https://mlsanity.com/paper/2005.09344/) (2020)
- [Towards Personalized and Semantic Retrieval: An End-to-End Solution for E-commerce Search via Embedding Learning](https://mlsanity.com/paper/2006.02282/) (2020)
- [Graph-based Multilingual Product Retrieval in E-commerce Search](https://mlsanity.com/paper/2105.02978/) (2021)
- [Equity2Vec: End-to-end Deep Learning Framework for Cross-sectional Asset Pricing](https://mlsanity.com/paper/1909.04497/) (2019)
- [BATS: A Spectral Biclustering Approach to Single Document Topic Modeling and Segmentation](https://mlsanity.com/paper/2008.02218/) (2020)
- [Lessons Learned Addressing Dataset Bias in Model-Based Candidate Generation at Twitter](https://mlsanity.com/paper/2105.09293/) (2021)

## Built on by (most cited first, 4 of 4)

- [AgentDR: Dynamic Recommendation with Implicit Item-Item Relations via LLM-based Agents](https://mlsanity.com/paper/2510.05598/) (2025)
- [Multi-modal Relational Item Representation Learning for Inferring Substitutable and Complementary Items](https://mlsanity.com/paper/2507.22268/) (2025)
- [A Survey of Reasoning for Substitution Relationships: Definitions, Methods, and Directions](https://mlsanity.com/paper/2404.08687/) (2024)
- [ULTRA:Urdu Language Transformer-based Recommendation Architecture](https://mlsanity.com/paper/2602.11836/) (2026)

## Abstract (arXiv)

The substitute-based recommendation is widely used in E-commerce to provide better alternatives to customers. However, existing research typically uses the customer behavior signals like co-view and view-but-purchase-another to capture the substitute relationship. Despite its intuitive soundness, we find that such an approach might ignore the functionality and characteristics of products. In this paper, we adapt substitute recommendation into language matching problem by taking product title description as model input to consider product functionality. We design a new transformation method to de-noise the signals derived from production data. In addition, we consider multilingual support from the engineering point of view. Our proposed end-to-end transformer-based model achieves both successes from offline and online experiments. The proposed model has been deployed in a large-scale E-commerce website for 11 marketplaces in 6 languages. Our proposed model is demonstrated to increase revenue by 19% based on an online A/B experiment.