# M2TRec: Metadata-aware Multi-task Transformer for Large-scale and Cold-start free Session-based Recommendations

arXiv: https://arxiv.org/abs/2209.11824 · HTML: https://mlsanity.com/paper/2209.11824/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 47 references that are in the map.
- First posted on arXiv 23 Sep 2022.

## Builds on (most cited first, 21 of 21)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [On the Properties of Neural Machine Translation: Encoder-Decoder Approaches](https://mlsanity.com/paper/1409.1259/) (2014)
- [An Overview of Multi-Task Learning in Deep Neural Networks](https://mlsanity.com/paper/1706.05098/) (2017)
- [Session-based Recommendation with Graph Neural Networks](https://mlsanity.com/paper/1811.00855/) (2018)
- [Neural Attentive Session-based Recommendation](https://mlsanity.com/paper/1711.04725/) (2017)
- [Recurrent Neural Networks with Top-k Gains for Session-based Recommendations](https://mlsanity.com/paper/1706.03847/) (2017)
- [Global Context Enhanced Graph Neural Networks for Session-based Recommendation](https://mlsanity.com/paper/2106.05081/) (2021)
- [RepeatNet: A Repeat Aware Neural Recommendation Machine for Session-based Recommendation](https://mlsanity.com/paper/1812.02646/) (2018)
- [Self-Supervised Graph Co-Training for Session-based Recommendation](https://mlsanity.com/paper/2108.10560/) (2021)
- [TAGNN: Target Attentive Graph Neural Networks for Session-based Recommendation](https://mlsanity.com/paper/2005.02844/) (2020)
- [Meta-Prod2Vec - Product Embeddings Using Side-Information for Recommendation](https://mlsanity.com/paper/1607.07326/) (2016)
- [Hierarchical Temporal Convolutional Networks for Dynamic Recommender Systems](https://mlsanity.com/paper/1904.04381/) (2019)
- [Exploiting Cross-Session Information for Session-based Recommendation with Graph Neural Networks](https://mlsanity.com/paper/2107.00852/) (2021)
- [Incorporating User Micro-behaviors and Item Knowledge into Multi-task Learning for Session-based Recommendation](https://mlsanity.com/paper/2006.06922/) (2020)
- [Graph-Enhanced Multi-Task Learning of Multi-Level Transition Dynamics for Session-based Recommendation](https://mlsanity.com/paper/2110.03996/) (2021)
- [Rank and Rate: Multi-task Learning for Recommender Systems](https://mlsanity.com/paper/1807.11698/) (2018)
- [Cold-start Sequential Recommendation via Meta Learner](https://mlsanity.com/paper/2012.05462/) (2020)
- [SIGIR 2021 E-Commerce Workshop Data Challenge](https://mlsanity.com/paper/2104.09423/) (2021)
- [Transformers with multi-modal features and post-fusion context for e-commerce session-based recommendation](https://mlsanity.com/paper/2107.05124/) (2021)
- [On the Importance of News Content Representation in Hybrid Neural Session-based Recommender Systems](https://mlsanity.com/paper/1907.07629/) (2019)

## Built on by (most cited first, 9 of 9)

- [Recent Developments in Recommender Systems: A Survey](https://mlsanity.com/paper/2306.12680/) (2023)
- [Code Recommendation for Open Source Software Developers](https://mlsanity.com/paper/2210.08332/) (2022)
- [A Multimodal Single-Branch Embedding Network for Recommendation in Cold-Start and Missing Modality Scenarios](https://mlsanity.com/paper/2409.17864/) (2024)
- [Enhancing User Intent Capture in Session-Based Recommendation with Attribute Patterns](https://mlsanity.com/paper/2312.16199/) (2023)
- [FINEST: Stabilizing Recommendations by Rank-Preserving Fine-Tuning](https://mlsanity.com/paper/2402.03481/) (2024)
- [Hierarchical Multi-Task Learning Framework for Session-based Recommendations](https://mlsanity.com/paper/2309.06533/) (2023)
- [Adversarial Text Rewriting for Text-aware Recommender Systems](https://mlsanity.com/paper/2408.00312/) (2024)
- [IntentRec: Predicting User Session Intent with Hierarchical Multi-Task Learning](https://mlsanity.com/paper/2408.05353/) (2024)
- [Let It Go? Not Quite: Addressing Item Cold Start in Sequential Recommendations with Content-Based Initialization](https://mlsanity.com/paper/2507.19473/) (2025)

## Abstract (arXiv)

Session-based recommender systems (SBRSs) have shown superior performance over conventional methods. However, they show limited scalability on large-scale industrial datasets since most models learn one embedding per item. This leads to a large memory requirement (of storing one vector per item) and poor performance on sparse sessions with cold-start or unpopular items. Using one public and one large industrial dataset, we experimentally show that state-of-the-art SBRSs have low performance on sparse sessions with sparse items. We propose M2TRec, a Metadata-aware Multi-task Transformer model for session-based recommendations. Our proposed method learns a transformation function from item metadata to embeddings, and is thus, item-ID free (i.e., does not need to learn one embedding per item). It integrates item metadata to learn shared representations of diverse item attributes. During inference, new or unpopular items will be assigned identical representations for the attributes they share with items previously observed during training, and thus will have similar representations with those items, enabling recommendations of even cold-start and sparse items. Additionally, M2TRec is trained in a multi-task setting to predict the next item in the session along with its primary category and subcategories. Our multi-task strategy makes the model converge faster and significantly improves the overall performance. Experimental results show significant performance gains using our proposed approach on sparse items on the two datasets.