seq2cause: One Autoregressive Backbone, Four Causal Discovery Tasks in Event Sequences
Not on the map. Why?
Abstract (arXiv)
Complex systems such as vehicles, patients, or genomes emit discrete event sequences whose operative question is causal, not predictive: which events cause which other events, and which cause higher-level outcomes such as failures or diseases? This question decomposes along two axes -- dependency type (event → event vs. event → outcome) and causal scope (single sequence vs. population) -- yielding four structurally distinct regimes with different identifiability conditions. No existing method addresses more than one, because all assume multi-stream structure with low vocabulary, and none scales beyond a few hundred event types. We present textscSeq2Cause, a unified framework that resolves all four regimes through a single shared primitive: a pretrained autoregressive model repurposed as an amortized conditional independence testing engine requiring no task-specific retraining. We establish a prediction--causality duality: the model's excess cross-entropy simultaneously bounds causal identification error across all four regimes, so that every improvement in next-token prediction tightens causal guarantees for free. On nonlinear SCMs (vocabularies up to 8,000 types) and real-world vehicle diagnostic logs (29K event types, 474 failure outcomes), textscSeq2Cause is the first method to populate all four regimes at scale with a single frozen backbone. Existing methods are either inapplicable, inaccurate, or computationally intractable in this setting.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 34 of its 73 references that are in the map. First posted on arXiv 25 Sep 2026.
Builds on
34 of 73 references.
References · 34
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
- 2017A Unified Approach to Interpreting Model Predictionscited by 6,555
- 2014Sequence to Sequence Learning with Neural Networkscited by 2,986
- 2017Learning Important Features Through Propagating Activation Differencescited by 1,458
- 2018DAGs with NO TEARS: Continuous Optimization for Structure Learningcited by 752
- 2024GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in Large Language Modelscited by 550
- 2019DAG-GNN: DAG Structure Learning with Graph Neural Networkscited by 331
- 2017Detecting causal associations in large nonlinear time series datasetscited by 244
- 2020Transformer Hawkes Processcited by 235
- 2020DYNOTEARS: Structure Learning from Time-Series Datacited by 200
- 2020On the Role of Sparsity and DAG Constraints for Learning Linear DAGscited by 191
22 more
- 2016Learning Granger Causality for Hawkes Processescited by 69
- 2023Causal Order: The Key to Leveraging Imperfect Experts in Causal Inferencecited by 48
- 2017Model-Powered Conditional Independence Testcited by 43
- 2023Causal Discovery from Temporal Data: An Overview and New Perspectivescited by 43
- 2019Conditional Independence Testing using Generative Adversarial Networkscited by 36
- 2020CAUSE: Learning Granger Causality from Event Sequences using Attribution Methodscited by 36
- 2023CUTS: Neural Causal Discovery from Irregular Time-Series Datacited by 33
- 2024Integrating Large Language Models in Causal Discovery: A Statistical Causal Approachcited by 32
- 2022Rhino: Deep Causal Temporal Relationship Learning With History-dependent Noisecited by 30
- 2023Root Cause Identification for Collective Anomalies in Time Series given an Acyclic Summary Causal Graph with Loopscited by 30
- 2023Causal Interpretation of Self-Attention in Pre-Trained Transformerscited by 30
- 2015Sparse and low-rank multivariate Hawkes processescited by 19
- 2021Causal Hidden Markov Model for Time Series Disease Forecastingcited by 19
- 2020BSODA: A Bipartite Scalable Framework for Online Disease Diagnosiscited by 14
- 2023Structural Hawkes Processes for Learning Causal Structure from Discrete-Time Event Sequencescited by 14
- 2024Local Causal Structure Learning in the Presence of Latent Variablescited by 11
- 2024Conditional Diffusion Models Based Conditional Independence Testingcited by 9
- 2024Harnessing Event Sensory Data for Error Pattern Prediction in Vehicles: A Language Model Approachcited by 6
- 2025Towards Practical Multi-label Causal Discovery in High-Dimensional Event Sequences via One-Shot Graph Aggregationcited by 4
- 2025One-Shot Multi-Label Causal Discovery in High-Dimensional Event Sequencescited by 4
- 2020Multi-label Causal Variable Discovery: Learning Common Causal Variables and Label-specific Causal Variablescited by 3
- 2024Hawkes Models And Their Applicationscited by 2
Built on by
Not cited yet.