mlsanityData to 7 Oct 2026

Towards Unified Approaches in Self-Supervised Event Stream Modeling: Progress and Prospects

arXiv 2502.04899 ↗First posted 7 Feb 2025cs.LG cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

The proliferation of digital interactions across diverse domains, such as healthcare, e-commerce, gaming, and finance, has resulted in the generation of vast volumes of event stream (ES) data. ES data comprises continuous sequences of timestamped events that encapsulate detailed contextual information relevant to each domain. While ES data holds significant potential for extracting actionable insights and enhancing decision-making, its effective utilization is hindered by challenges such as the scarcity of labeled data and the fragmented nature of existing research efforts. Self-Supervised Learning (SSL) has emerged as a promising paradigm to address these challenges by enabling the extraction of meaningful representations from unlabeled ES data. In this survey, we systematically review and synthesize SSL methodologies tailored for ES modeling across multiple domains, bridging the gaps between domain-specific approaches that have traditionally operated in isolation. We present a comprehensive taxonomy of SSL techniques, encompassing both predictive and contrastive paradigms, and analyze their applicability and effectiveness within different application contexts. Furthermore, we identify critical gaps in current research and propose a future research agenda aimed at developing scalable, domain-agnostic SSL frameworks for ES modeling. By unifying disparate research efforts and highlighting cross-domain synergies, this survey aims to accelerate innovation, improve reproducibility, and expand the applicability of SSL to diverse real-world ES challenges.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 75 of its 110 references that are in the map. First posted on arXiv 7 Feb 2025.

Builds on

75 of 110 references.

References · 75

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  5. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  6. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  7. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  8. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  9. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  10. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  11. 2020Longformer: The Long-Document Transformercited by 3,362
  12. 2020Exploring Simple Siamese Representation Learningcited by 2,853
38 more
  1. 2021Barlow Twins: Self-Supervised Learning via Redundancy Reductioncited by 1,818
  2. 2021VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learningcited by 1,013
  3. 2023Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecturecited by 982
  4. 2019BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformercited by 952
  5. 2015Image-based Recommendations on Styles and Substitutescited by 702
  6. 2020Self-supervised Learning: Generative or Contrastivecited by 693
  7. 2020The Future of Digital Health with Federated Learningcited by 625
  8. 2024Revisiting Feature Prediction for Learning Visual Representations from Videocited by 490
  9. 2018Personalized Top-N Sequential Recommendation via Convolutional Sequence Embeddingcited by 473
  10. 2022Recommendation as Language Processing (RLP): A Unified Pretrain, Personalized Prompt & Predict Paradigm (P5)cited by 396
  11. 2020Med-BERT: pre-trained contextualized embeddings on large-scale structured electronic health records for disease predictioncited by 368
  12. 2020S^3-Rec: Self-Supervised Learning for Sequential Recommendation with Mutual Information Maximizationcited by 311
  13. 2019Representation Learning for Dynamic Graphs: A Surveycited by 260
  14. 2019BEHRT: Transformer for Electronic Health Recordscited by 258
  15. 2023A Cookbook of Self-Supervised Learningcited by 248
  16. 2021Multi-Time Attention Networks for Irregularly Sampled Time Seriescited by 234
  17. 2024Foundation Models for Time Series Analysis: A Tutorial and Surveycited by 234
  18. 2020Transformer Hawkes Processcited by 230
  19. 2017Deep EHR: A Survey of Recent Advances in Deep Learning Techniques for Electronic Health Record (EHR) Analysiscited by 145
  20. 2019Intensity-Free Learning of Temporal Point Processescited by 138
  21. 2020Foundations and modelling of dynamic networks using Dynamic Graph Neural Networks: A surveycited by 123
  22. 2019MIMIC-Extract: A Data Extraction, Preprocessing, and Representation Pipeline for MIMIC-IIIcited by 121
  23. 2021Contrastive Learning for Representation Degeneration Problem in Sequential Recommendationcited by 121
  24. 2018A Simple Convolutional Generative Network for Next Item Recommendationcited by 120
  25. 2022Towards Universal Sequence Representation Learning for Recommender Systemscited by 120
  26. 2019Fully Neural Network based Model for General Temporal Point Processescited by 119
  27. 2019Sequential Recommender Systems: Challenges, Progress and Prospectscited by 115
  28. 2021Neural Temporal Point Processes: A Reviewcited by 113
  29. 2021BYOL for Audio: Self-Supervised Learning for General-Purpose Audio Representationcited by 106
  30. 2023EHRSHOT: An EHR Benchmark for Few-Shot Evaluation of Foundation Modelscited by 101
  31. 2019A Survey on Session-based Recommender Systemscited by 97
  32. 2023Self-Supervised Learning for Time Series Analysis: Taxonomy, Progress, and Prospectscited by 97
  33. 2022Self-Supervised Learning for Recommender Systems: A Surveycited by 95
  34. 2020Neural Spatio-Temporal Point Processescited by 92
  35. 2021Transformer Embeddings of Irregularly Spaced Events and Their Participantscited by 90
  36. 2015SEISMIC: A Self-Exciting Point Process Model for Predicting Tweet Popularitycited by 76
  37. 2023MOTOR: A Time-To-Event Foundation Model For Structured Medical Recordscited by 75
  38. 2023EasyTPP: Towards Open Benchmarking Temporal Point Processescited by 63

50 most cited of 75 All 75 (JSON)

Built on by

Not cited yet.