# Towards Lifelong Learning Embeddings: An Algorithmic Approach to Dynamically Extend Embeddings

arXiv: https://arxiv.org/abs/2408.14118 · HTML: https://mlsanity.com/paper/2408.14118/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 32 references that are in the map.
- First posted on arXiv 26 Aug 2024.

## Builds on (most cited first, 11 of 11)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [iCaRL: Incremental Classifier and Representation Learning](https://mlsanity.com/paper/1611.07725/) (2016)
- [Gradient Episodic Memory for Continual Learning](https://mlsanity.com/paper/1706.08840/) (2017)
- [BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer](https://mlsanity.com/paper/1904.06690/) (2019)
- [Deep Interest Evolution Network for Click-Through Rate Prediction](https://mlsanity.com/paper/1809.03672/) (2018)
- [ATRank: An Attention-Based User Behavior Modeling Framework for Recommendation](https://mlsanity.com/paper/1711.06632/) (2017)
- [Meta-Prod2Vec - Product Embeddings Using Side-Information for Recommendation](https://mlsanity.com/paper/1607.07326/) (2016)
- [Modeling Users' Contextualized Page-wise Feedback for Click-Through Rate Prediction in E-commerce Search](https://mlsanity.com/paper/2203.15542/) (2022)
- [Predicting purchasing intent: Automatic Feature Learning using Recurrent Neural Networks](https://mlsanity.com/paper/1807.08207/) (2018)

## Abstract (arXiv)

The rapid evolution of technology has transformed business operations and customer interactions worldwide, with personalization emerging as a key opportunity for e-commerce companies to engage customers more effectively. The application of machine learning, particularly that of deep learning models, has gained significant traction due to its ability to rapidly recognize patterns in large datasets, thereby offering numerous possibilities for personalization. These models use embeddings to map discrete information, such as product IDs, into a latent vector space, a method increasingly popular in recent years. However, e-commerce's dynamic nature, characterized by frequent new product introductions, poses challenges for these embeddings, which typically require fixed dimensions and inputs, leading to the need for periodic retraining from scratch. This paper introduces a modular algorithm that extends embedding input size while preserving learned knowledge, addressing the challenges posed by e-commerce's dynamism. The proposed algorithm also incorporates strategies to mitigate the cold start problem associated with new products. The results of initial experiments suggest that this method outperforms traditional embeddings.