Pretext Training Algorithms for Event Sequence Data
Not on the map. Why?
Abstract (arXiv)
Pretext training followed by task-specific fine-tuning has been a successful approach in vision and language domains. This paper proposes a self-supervised pretext training framework tailored to event sequence data. We introduce a novel alignment verification task that is specialized to event sequences, building on good practices in masked reconstruction and contrastive learning. Our pretext tasks unlock foundational representations that are generalizable across different down-stream tasks, including next-event prediction for temporal point process models, event sequence classification, and missing event interpolation. Experiments on popular public benchmarks demonstrate the potential of the proposed method across different tasks and data domains.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 35 of its 48 references that are in the map. First posted on arXiv 16 Feb 2024.
Builds on
35 of 48 references.
References · 35
- 2017Attention Is All You Needcited by 59,932
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2020Language Models are Few-Shot Learnerscited by 32,199
- 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
- 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
- 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
- 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
- 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
- 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
- 2018Neural Ordinary Differential Equationscited by 3,733
- 2020What Makes for Good Views for Contrastive Learning?cited by 843
23 more
- 2023Time-LLM: Time Series Forecasting by Reprogramming Large Language Modelscited by 680
- 2019End-to-End Learning of Visual Representations from Uncurated Instructional Videoscited by 506
- 2023Large Language Models Are Zero-Shot Time Series Forecasterscited by 491
- 2021TS2Vec: Towards Universal Representation of Time Seriescited by 463
- 2020A Transformer-based Framework for Multivariate Time Series Representation Learningcited by 449
- 2019Predicting Dynamic Embedding Trajectory in Temporal Interaction Networkscited by 443
- 2019Unsupervised Scalable Representation Learning for Multivariate Time Seriescited by 267
- 2020UniVL: A Unified Video and Language Pre-Training Model for Multimodal Understanding and Generationcited by 258
- 2021Multi-Time Attention Networks for Irregularly Sampled Time Seriescited by 234
- 2021Unsupervised Representation Learning for Time Series with Temporal Neighborhood Codingcited by 223
- 2019Intensity-Free Learning of Temporal Point Processescited by 138
- 2021Neural Temporal Point Processes: A Reviewcited by 113
- 2021Neural Flows: Efficient Alternative to Neural ODEscited by 99
- 2021Transformer Embeddings of Irregularly Spaced Events and Their Participantscited by 90
- 2018Community Interaction and Conflict on the Webcited by 72
- 2023EasyTPP: Towards Open Benchmarking Temporal Point Processescited by 63
- 2018Perfect match: Improved cross-modal embeddings for audio-visual synchronisationcited by 60
- 2019Self-attention with Functional Time Representation Learningcited by 52
- 2023Realistic Synthetic Financial Transactions for Anti-Money Laundering Modelscited by 47
- 2019Imputing Missing Events in Continuous-Time Event Streamscited by 43
- 2019A Variational Auto-Encoder Model for Stochastic Point Processescited by 30
- 2023Meta Temporal Point Processescited by 16
- 2023What Constitutes Good Contrastive Learning in Time-Series Forecasting?cited by 2
Built on by
Citing papers per year: 2026: 1.
1 paper.