# ContrastVAE: Contrastive Variational AutoEncoder for Sequential Recommendation

arXiv: https://arxiv.org/abs/2209.00456 · HTML: https://mlsanity.com/paper/2209.00456/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 46 references that are in the map.
- First posted on arXiv 27 Aug 2022.

## Builds on (most cited first, 30 of 32)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [SimCSE: Simple Contrastive Learning of Sentence Embeddings](https://mlsanity.com/paper/2104.08821/) (2021)
- [Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hypersphere](https://mlsanity.com/paper/2005.10242/) (2020)
- [Learning deep representations by mutual information estimation and maximization](https://mlsanity.com/paper/1808.06670/) (2018)
- [Self-Attentive Sequential Recommendation](https://mlsanity.com/paper/1808.09781/) (2018)
- [LightGCN: Simplifying and Powering Graph Convolution Network for Recommendation](https://mlsanity.com/paper/2002.02126/) (2020)
- [BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer](https://mlsanity.com/paper/1904.06690/) (2019)
- [Image-based Recommendations on Styles and Substitutes](https://mlsanity.com/paper/1506.04757/) (2015)
- [Session-based Recommendations with Recurrent Neural Networks](https://mlsanity.com/paper/1511.06939/) (2015)
- [MINE: Mutual Information Neural Estimation](https://mlsanity.com/paper/1801.04062/) (2018)
- [On Variational Bounds of Mutual Information](https://mlsanity.com/paper/1905.06922/) (2019)
- [Variational Dropout and the Local Reparameterization Trick](https://mlsanity.com/paper/1506.02557/) (2015)
- [S^3-Rec: Self-Supervised Learning for Sequential Recommendation with Mutual Information Maximization](https://mlsanity.com/paper/2008.07873/) (2020)
- [Variational Autoencoders for Collaborative Filtering](https://mlsanity.com/paper/1802.05814/) (2018)
- [Self-supervised Graph Learning for Recommendation](https://mlsanity.com/paper/2010.10783/) (2020)
- [Fusing Similarity Models with Markov Chains for Sparse Sequential Recommendation](https://mlsanity.com/paper/1609.09152/) (2016)
- [Don't Blame the ELBO! A Linear VAE Perspective on Posterior Collapse](https://mlsanity.com/paper/1911.02469/) (2019)
- [Contrastive Learning for Representation Degeneration Problem in Sequential Recommendation](https://mlsanity.com/paper/2110.05730/) (2021)
- [RecVAE: a New Variational Autoencoder for Top-N Recommendations with Implicit Feedback](https://mlsanity.com/paper/1912.11160/) (2019)
- [Contrastive Self-supervised Sequential Recommendation with Robust Augmentation](https://mlsanity.com/paper/2108.06479/) (2021)
- [Augmenting Sequential Recommendation with Pseudo-Prior Items via Reversely Pre-training Transformer](https://mlsanity.com/paper/2105.00522/) (2021)
- [Sequential Recommendation via Stochastic Self-Attention](https://mlsanity.com/paper/2201.06035/) (2022)
- [Contrastively Disentangled Sequential Variational Autoencoder](https://mlsanity.com/paper/2110.12091/) (2021)
- [Sequential Variational Autoencoders for Collaborative Filtering](https://mlsanity.com/paper/1811.09975/) (2018)
- [Adversarial and Contrastive Variational Autoencoder for Sequential Recommendation](https://mlsanity.com/paper/2103.10693/) (2021)
- [Modeling Sequences as Distributions with Uncertainty for Sequential Recommendation](https://mlsanity.com/paper/2106.06165/) (2021)
- [DSKReG: Differentiable Sampling on Knowledge Graph for Recommendation with Relational GNN](https://mlsanity.com/paper/2108.11883/) (2021)

## Built on by (most cited first, 19 of 19)

- [A Review of Modern Recommender Systems Using Generative Models (Gen-RecSys)](https://mlsanity.com/paper/2404.00579/) (2024)
- [Conditional Denoising Diffusion for Sequential Recommendation](https://mlsanity.com/paper/2304.11433/) (2023)
- [MAP: A Model-agnostic Pretraining Framework for Click-through Rate Prediction](https://mlsanity.com/paper/2308.01737/) (2023)
- [Personalized Federated Collaborative Filtering: A Variational AutoEncoder Approach](https://mlsanity.com/paper/2408.08931/) (2024)
- [FedDCSR: Federated Cross-domain Sequential Recommendation via Disentangled Representation Learning](https://mlsanity.com/paper/2309.08420/) (2023)
- [A Comprehensive Survey on Self-Supervised Learning for Recommendation](https://mlsanity.com/paper/2404.03354/) (2024)
- [Unleashing the Potential of Two-Tower Models: Diffusion-Based Cross-Interaction for Large-Scale Matching](https://mlsanity.com/paper/2502.20687/) (2025)
- [C2VAE: Gaussian Copula-based VAE Differing Disentangled from Coupled Representations with Contrastive Posterior](https://mlsanity.com/paper/2309.13303/) (2023)
- [Knowledge Graph Context-Enhanced Diversified Recommendation](https://mlsanity.com/paper/2310.13253/) (2023)
- [Variational Self-Supervised Learning](https://mlsanity.com/paper/2504.04318/) (2025)
- [Contrast-augmented Diffusion Model with Fine-grained Sequence Alignment for Markup-to-Image Generation](https://mlsanity.com/paper/2308.01147/) (2023)
- [Contrastive Learning Method for Sequential Recommendation based on Multi-Intention Disentanglement](https://mlsanity.com/paper/2404.18214/) (2024)
- [Enhancing Recommendation with Denoising Auxiliary Task](https://mlsanity.com/paper/2409.17402/) (2024)
- [Multi-Perspective Attention Mechanism for Bias-Aware Sequential Recommendation](https://mlsanity.com/paper/2504.05323/) (2025)
- [Large-scale User Game Lifecycle Representation Learning](https://mlsanity.com/paper/2510.15412/) (2025)
- [Structured Contrastive Learning for Interpretable Latent Representations](https://mlsanity.com/paper/2511.14920/) (2025)
- [Improving Cardiac Risk Prediction Using Data Generation Techniques](https://mlsanity.com/paper/2512.20669/) (2025)
- [From Agnostic to Specific: Latent Preference Diffusion for Multi-Behavior Sequential Recommendation](https://mlsanity.com/paper/2602.23132/) (2026)
- [Debiasing Sequential Recommendation with Time-aware Inverse Propensity Scoring](https://mlsanity.com/paper/2603.04986/) (2026)

## Abstract (arXiv)

Aiming at exploiting the rich information in user behaviour sequences, sequential recommendation has been widely adopted in real-world recommender systems. However, current methods suffer from the following issues: 1) sparsity of user-item interactions, 2) uncertainty of sequential records, 3) long-tail items. In this paper, we propose to incorporate contrastive learning into the framework of Variational AutoEncoders to address these challenges simultaneously. Firstly, we introduce ContrastELBO, a novel training objective that extends the conventional single-view ELBO to two-view case and theoretically builds a connection between VAE and contrastive learning from a two-view perspective. Then we propose Contrastive Variational AutoEncoder (ContrastVAE in short), a two-branched VAE model with contrastive regularization as an embodiment of ContrastELBO for sequential recommendation. We further introduce two simple yet effective augmentation strategies named model augmentation and variational augmentation to create a second view of a sequence and thus making contrastive learning possible. Experiments on four benchmark datasets demonstrate the effectiveness of ContrastVAE and the proposed augmentation methods. Codes are available at https://github.com/YuWang-1024/ContrastVAE