# CoLES: Contrastive Learning for Event Sequences with Self-Supervision

arXiv: https://arxiv.org/abs/2002.08232 · HTML: https://mlsanity.com/paper/2002.08232/

## Key facts

- Cited by 32 papers in the map (data through 7 Oct 2026).
- First posted on arXiv 19 Feb 2020.

## Built on by (most cited first, 30 of 32)

- [LightPath: Lightweight and Scalable Path Representation Learning](https://mlsanity.com/paper/2307.10171/) (2023)
- [Towards a Foundation Purchasing Model: Pretrained Generative Autoregression on Transaction Sequences](https://mlsanity.com/paper/2401.01641/) (2024)
- [HoTPP Benchmark: Are We Good at the Long Horizon Events Forecasting?](https://mlsanity.com/paper/2406.14341/) (2024)
- [Multimodal Banking Dataset: Understanding Client Needs through Event Sequences](https://mlsanity.com/paper/2409.17587/) (2024)
- [MLEM: Generative and Contrastive Learning as Distinct Modalities for Event Sequences](https://mlsanity.com/paper/2401.15935/) (2024)
- [EBES: Easy Benchmarking for Event Sequences](https://mlsanity.com/paper/2410.03399/) (2024)
- [Learning Transactions Representations for Information Management in Banks: Mastering Local, Global, and External Knowledge](https://mlsanity.com/paper/2404.02047/) (2024)
- [Label Attention Network for Temporal Sets Prediction: You Were Looking at a Wrong Self-Attention](https://mlsanity.com/paper/2303.00280/) (2023)
- [Uniting contrastive and generative learning for event sequences models](https://mlsanity.com/paper/2408.09995/) (2024)
- [LATTE: Learning Aligned Transactions and Textual Embeddings for Bank Clients](https://mlsanity.com/paper/2508.10021/) (2025)
- [Open Banking Foundational Model: Learning Language Representations from Few Financial Transactions](https://mlsanity.com/paper/2511.12154/) (2025)
- [Topological Metric for Unsupervised Embedding Quality Evaluation](https://mlsanity.com/paper/2512.15285/) (2025)
- [The Long Tail of Context: Does it Exist and Matter?](https://mlsanity.com/paper/2210.01023/) (2022)
- [Hiding Backdoors within Event Sequence Data via Poisoning Attacks](https://mlsanity.com/paper/2308.10201/) (2023)
- [Locally Differentially Private Embedding Models in Distributed Fraud Prevention Systems](https://mlsanity.com/paper/2401.02450/) (2024)
- [ESQA: Event Sequences Question Answering](https://mlsanity.com/paper/2407.12833/) (2024)
- [Embedding-Aware Feature Discovery: Bridging Latent Representations and Interpretable Features in Event Sequences](https://mlsanity.com/paper/2603.15713/) (2026)
- [Continuous-time convolutions model of event sequences](https://mlsanity.com/paper/2302.06247/) (2023)
- [Designing an attack-defense game: how to increase robustness of financial transaction models via a competition](https://mlsanity.com/paper/2308.11406/) (2023)
- [Self-Attention Mechanism in Multimodal Context for Banking Transaction Flow](https://mlsanity.com/paper/2410.08243/) (2024)
- [Towards Unified Approaches in Self-Supervised Event Stream Modeling: Progress and Prospects](https://mlsanity.com/paper/2502.04899/) (2025)
- [Looking around you: external information enhances representations for event sequences](https://mlsanity.com/paper/2502.10205/) (2025)
- [How to model Human Actions distribution with Event Sequence Data](https://mlsanity.com/paper/2510.05856/) (2025)
- [Automated Evolutionary Optimization for Resource-Efficient Neural Network Training](https://mlsanity.com/paper/2510.09566/) (2025)
- [Beyond Isolated Clients: Integrating Graph-Based Embeddings into Event Sequence Models](https://mlsanity.com/paper/2604.09085/) (2026)
- [When Losses Align: Gradient-Based Composite Loss Weighting for Efficient Pretraining](https://mlsanity.com/paper/2605.07756/) (2026)
- [SOHET: Sequence Of Heterogeneous Events Transformer with Self-Supervised Pre-Training](https://mlsanity.com/paper/2606.21356/) (2026)
- [A Foundation Model for Multimodal Event Sequences in Financial Applications](https://mlsanity.com/paper/2607.09955/) (2026)
- [User-Centric Modeling of Transactional Sequences with Explainable State Space Models](https://mlsanity.com/paper/2607.20228/) (2026)
- [MINT: A Universal Zero-Shot Predictor for Transaction Data](https://mlsanity.com/paper/2608.14198/) (2026)

## Abstract (arXiv)

We address the problem of self-supervised learning on discrete event sequences generated by real-world users. Self-supervised learning incorporates complex information from the raw data in low-dimensional fixed-length vector representations that could be easily applied in various downstream machine learning tasks. In this paper, we propose a new method "CoLES", which adapts contrastive learning, previously used for audio and computer vision domains, to the discrete event sequences domain in a self-supervised setting. We deployed CoLES embeddings based on sequences of transactions at the large European financial services company. Usage of CoLES embeddings significantly improves the performance of the pre-existing models on downstream tasks and produces significant financial gains, measured in hundreds of millions of dollars yearly. We also evaluated CoLES on several public event sequences datasets and showed that CoLES representations consistently outperform other methods on different downstream tasks.