# MLEM: Generative and Contrastive Learning as Distinct Modalities for Event Sequences

arXiv: https://arxiv.org/abs/2401.15935 · HTML: https://mlsanity.com/paper/2401.15935/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 43 references that are in the map.
- First posted on arXiv 29 Jan 2024.

## Builds on (most cited first, 23 of 23)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [Estimating the intrinsic dimension of datasets by a minimal neighborhood information](https://mlsanity.com/paper/1803.06992/) (2018)
- [Unsupervised Representation Learning for Time Series with Temporal Neighborhood Coding](https://mlsanity.com/paper/2106.00750/) (2021)
- [Latent ODEs for Irregularly-Sampled Time Series](https://mlsanity.com/paper/1907.03907/) (2019)
- [Contrast with Reconstruct: Contrastive 3D Representation Learning Guided by Generative Pretraining](https://mlsanity.com/paper/2302.02318/) (2023)
- [Learning Optimal Representations with the Decodable Information Bottleneck](https://mlsanity.com/paper/2009.12789/) (2020)
- [CoLES: Contrastive Learning for Event Sequences with Self-Supervision](https://mlsanity.com/paper/2002.08232/) (2020)
- [Exploring Generative Neural Temporal Point Process](https://mlsanity.com/paper/2208.01874/) (2022)
- [Adversarial Attacks on Deep Models for Financial Transaction Records](https://mlsanity.com/paper/2106.08361/) (2021)
- [Generative Contrastive Graph Learning for Recommendation](https://mlsanity.com/paper/2307.05100/) (2023)
- [An Empirical Study: Extensive Deep Temporal Point Process](https://mlsanity.com/paper/2110.09823/) (2021)
- [Hybrid Generative-Contrastive Representation Learning](https://mlsanity.com/paper/2106.06162/) (2021)
- [SeqNAS: Neural Architecture Search for Event Sequence Classification](https://mlsanity.com/paper/2401.03246/) (2024)
- [Modeling Events and Interactions through Temporal Processes -- A Survey](https://mlsanity.com/paper/2303.06067/) (2023)
- [IMAD: IMage-Augmented multi-modal Dialogue](https://mlsanity.com/paper/2305.10512/) (2023)

## Built on by (most cited first, 6 of 6)

- [EBES: Easy Benchmarking for Event Sequences](https://mlsanity.com/paper/2410.03399/) (2024)
- [Uniting contrastive and generative learning for event sequences models](https://mlsanity.com/paper/2408.09995/) (2024)
- [ESQA: Event Sequences Question Answering](https://mlsanity.com/paper/2407.12833/) (2024)
- [Leveraging LLM Parametric Knowledge for Fact Checking without Retrieval](https://mlsanity.com/paper/2603.05471/) (2026)
- [SOHET: Sequence Of Heterogeneous Events Transformer with Self-Supervised Pre-Training](https://mlsanity.com/paper/2606.21356/) (2026)
- [A Foundation Model for Multimodal Event Sequences in Financial Applications](https://mlsanity.com/paper/2607.09955/) (2026)

## Abstract (arXiv)

This study explores the application of self-supervised learning techniques for event sequences. It is a key modality in various applications such as banking, e-commerce, and healthcare. However, there is limited research on self-supervised learning for event sequences, and methods from other domains like images, texts, and speech may not easily transfer. To determine the most suitable approach, we conduct a detailed comparative analysis of previously identified best-performing methods. We find that neither the contrastive nor generative method is superior. Our assessment includes classifying event sequences, predicting the next event, and evaluating embedding quality. These results further highlight the potential benefits of combining both methods. Given the lack of research on hybrid models in this domain, we initially adapt the baseline model from another domain. However, upon observing its underperformance, we develop a novel method called the Multimodal-Learning Event Model (MLEM). MLEM treats contrastive learning and generative modeling as distinct yet complementary modalities, aligning their embeddings. The results of our study demonstrate that combining contrastive and generative approaches into one procedure with MLEM achieves superior performance across multiple metrics.