ESQA: Event Sequences Question Answering
Not on the map. Why?
Abstract (arXiv)
Event sequences (ESs) arise in many practical domains including finance, retail, social networks, and healthcare. In the context of machine learning, event sequences can be seen as a special type of tabular data with annotated timestamps. Despite the importance of ESs modeling and analysis, little effort was made in adapting large language models (LLMs) to the ESs domain. In this paper, we highlight the common difficulties of ESs processing and propose a novel solution capable of solving multiple downstream tasks with little or no finetuning. In particular, we solve the problem of working with long sequences and improve time and numeric features processing. The resulting method, called ESQA, effectively utilizes the power of LLMs and, according to extensive experiments, achieves state-of-the-art results in the ESs domain.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 42 of its 97 references that are in the map. First posted on arXiv 3 Jul 2024.
Builds on
42 of 97 references.
References · 42
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2020Language Models are Few-Shot Learnerscited by 32,199
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
- 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
- 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
- 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
- 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
- 2020Longformer: The Long-Document Transformercited by 3,362
- 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
- 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
- 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,772
30 more
- 2021Barlow Twins: Self-Supervised Learning via Redundancy Reductioncited by 1,818
- 2017Decoupled Weight Decay Regularizationcited by 1,722
- 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,235
- 2023One Fits All:Power General Time Series Analysis by Pretrained LMcited by 615
- 2023Language Is Not All You Need: Aligning Perception with Language Modelscited by 543
- 2023Large Language Models Are Zero-Shot Time Series Forecasterscited by 491
- 2020TaBERT: Pretraining for Joint Understanding of Textual and Tabular Datacited by 395
- 2022TabLLM: Few-shot Classification of Tabular Data with Large Language Modelscited by 273
- 2023AI Alignment: A Comprehensive Surveycited by 256
- 2021Multi-Time Attention Networks for Irregularly Sampled Time Seriescited by 234
- 2023Evaluating the Moral Beliefs Encoded in LLMscited by 211
- 2024BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystackcited by 211
- 2021Pretrained Transformers as Universal Computation Enginescited by 162
- 2022LIFT: Language-Interfaced Fine-Tuning for Non-Language Machine Learning Taskscited by 146
- 2019Intensity-Free Learning of Temporal Point Processescited by 138
- 2020Moral Stories: Situated Reasoning about Norms, Intents, Actions, and their Consequencescited by 133
- 2023Grounding Language Models to Images for Multimodal Inputs and Outputscited by 129
- 2024Large Language Models for Time Series: A Surveycited by 113
- 2021TABBIE: Pretrained Representations of Tabular Datacited by 111
- 2022TableFormer: Robust Transformer Modeling for Table-Text Encodingcited by 90
- 2020Tabular Transformers for Modeling Multivariate Time Seriescited by 77
- 2023EasyTPP: Towards Open Benchmarking Temporal Point Processescited by 63
- 2023Event Stream GPT: A Data Pre-processing and Modeling Library for Generative, Pre-trained Transformers over Continuous-time Sequences of Complex Eventscited by 46
- 2020CoLES: Contrastive Learning for Event Sequences with Self-Supervisioncited by 32
- 2023Generalized Logit Adjustment: Calibrating Fine-tuned Models by Removing Label Bias in Foundation Modelscited by 32
- 2018Perceive Your Users in Depth: Learning Universal User Representations from Multiple E-commerce Taskscited by 24
- 2024HoTPP Benchmark: Are We Good at the Long Horizon Events Forecasting?cited by 11
- 2024MLEM: Generative and Contrastive Learning as Distinct Modalities for Event Sequencescited by 6
- 2020Crime Prediction Using Spatio-Temporal Datacited by 5
- 2021Towards Explainable Artificial Intelligence in Banking and Financial Servicescited by 1
Built on by
Citing papers per year: 2025: 1.
1 paper.