# Theoretical Understandings of Product Embedding for E-commerce Machine Learning

arXiv: https://arxiv.org/abs/2102.12029 · HTML: https://mlsanity.com/paper/2102.12029/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 32 references that are in the map.
- First posted on arXiv 24 Feb 2021.

## Builds on (most cited first, 16 of 16)

- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [node2vec: Scalable Feature Learning for Networks](https://mlsanity.com/paper/1607.00653/) (2016)
- [Wide & Deep Learning for Recommender Systems](https://mlsanity.com/paper/1606.07792/) (2016)
- [Are We Really Making Much Progress? A Worrying Analysis of Recent Neural Recommendation Approaches](https://mlsanity.com/paper/1907.06902/) (2019)
- [Embedding-based Retrieval in Facebook Search](https://mlsanity.com/paper/2006.11632/) (2020)
- [Neural Collaborative Filtering vs. Matrix Factorization Revisited](https://mlsanity.com/paper/2005.09683/) (2020)
- [word2vec Explained: deriving Mikolov et al.'s negative-sampling word-embedding method](https://mlsanity.com/paper/1402.3722/) (2014)
- [Billion-scale Commodity Embedding for E-commerce Recommendation in Alibaba](https://mlsanity.com/paper/1803.02349/) (2018)
- [Item2Vec: Neural Item Embedding for Collaborative Filtering](https://mlsanity.com/paper/1603.04259/) (2016)
- [Analogies Explained: Towards Understanding Word Embeddings](https://mlsanity.com/paper/1901.09813/) (2019)
- [Product Knowledge Graph Embedding for E-commerce](https://mlsanity.com/paper/1911.12481/) (2019)
- [Meta-Prod2Vec - Product Embeddings Using Side-Information for Recommendation](https://mlsanity.com/paper/1607.07326/) (2016)
- [E-commerce in Your Inbox: Product Recommendations at Scale](https://mlsanity.com/paper/1606.07154/) (2016)
- [Knowledge-aware Complementary Product Representation Learning](https://mlsanity.com/paper/1904.12574/) (2019)
- [What the Vec? Towards Probabilistically Grounded Embeddings](https://mlsanity.com/paper/1805.12164/) (2018)

## Built on by (most cited first, 4 of 4)

- [Rethinking Neural vs. Matrix-Factorization Collaborative Filtering: the Theoretical Perspectives](https://mlsanity.com/paper/2110.12141/) (2021)
- [It's Enough: Relaxing Diagonal Constraints in Linear Autoencoders for Recommendation](https://mlsanity.com/paper/2305.12922/) (2023)
- [GRAFENNE: Learning on Graphs with Heterogeneous and Dynamic Feature Sets](https://mlsanity.com/paper/2306.03447/) (2023)
- [Pretrained Embeddings for E-commerce Machine Learning: When it Fails and Why?](https://mlsanity.com/paper/2304.04330/) (2023)

## Abstract (arXiv)

Product embeddings have been heavily investigated in the past few years, serving as the cornerstone for a broad range of machine learning applications in e-commerce. Despite the empirical success of product embeddings, little is known on how and why they work from the theoretical standpoint. Analogous results from the natural language processing (NLP) often rely on domain-specific properties that are not transferable to the e-commerce setting, and the downstream tasks often focus on different aspects of the embeddings. We take an e-commerce-oriented view of the product embeddings and reveal a complete theoretical view from both the representation learning and the learning theory perspective. We prove that product embeddings trained by the widely-adopted skip-gram negative sampling algorithm and its variants are sufficient dimension reduction regarding a critical product relatedness measure. The generalization performance in the downstream machine learning task is controlled by the alignment between the embeddings and the product relatedness measure. Following the theoretical discoveries, we conduct exploratory experiments that supports our theoretical insights for the product embeddings.