# seq2cause: One Autoregressive Backbone, Four Causal Discovery Tasks in Event Sequences

arXiv: https://arxiv.org/abs/2609.31801 · HTML: https://mlsanity.com/paper/2609.31801/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 34 of its 73 references that are in the map.
- First posted on arXiv 25 Sep 2026.

## Builds on (most cited first, 30 of 34)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [Learning Important Features Through Propagating Activation Differences](https://mlsanity.com/paper/1704.02685/) (2017)
- [DAGs with NO TEARS: Continuous Optimization for Structure Learning](https://mlsanity.com/paper/1803.01422/) (2018)
- [GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in Large Language Models](https://mlsanity.com/paper/2410.05229/) (2024)
- [DAG-GNN: DAG Structure Learning with Graph Neural Networks](https://mlsanity.com/paper/1904.10098/) (2019)
- [Detecting causal associations in large nonlinear time series datasets](https://mlsanity.com/paper/1702.07007/) (2017)
- [Transformer Hawkes Process](https://mlsanity.com/paper/2002.09291/) (2020)
- [DYNOTEARS: Structure Learning from Time-Series Data](https://mlsanity.com/paper/2002.00498/) (2020)
- [On the Role of Sparsity and DAG Constraints for Learning Linear DAGs](https://mlsanity.com/paper/2006.10201/) (2020)
- [Learning Granger Causality for Hawkes Processes](https://mlsanity.com/paper/1602.04511/) (2016)
- [Causal Order: The Key to Leveraging Imperfect Experts in Causal Inference](https://mlsanity.com/paper/2310.15117/) (2023)
- [Model-Powered Conditional Independence Test](https://mlsanity.com/paper/1709.06138/) (2017)
- [Causal Discovery from Temporal Data: An Overview and New Perspectives](https://mlsanity.com/paper/2303.10112/) (2023)
- [Conditional Independence Testing using Generative Adversarial Networks](https://mlsanity.com/paper/1907.04068/) (2019)
- [CAUSE: Learning Granger Causality from Event Sequences using Attribution Methods](https://mlsanity.com/paper/2002.07906/) (2020)
- [CUTS: Neural Causal Discovery from Irregular Time-Series Data](https://mlsanity.com/paper/2302.07458/) (2023)
- [Integrating Large Language Models in Causal Discovery: A Statistical Causal Approach](https://mlsanity.com/paper/2402.01454/) (2024)
- [Rhino: Deep Causal Temporal Relationship Learning With History-dependent Noise](https://mlsanity.com/paper/2210.14706/) (2022)
- [Root Cause Identification for Collective Anomalies in Time Series given an Acyclic Summary Causal Graph with Loops](https://mlsanity.com/paper/2303.04038/) (2023)
- [Causal Interpretation of Self-Attention in Pre-Trained Transformers](https://mlsanity.com/paper/2310.20307/) (2023)
- [Sparse and low-rank multivariate Hawkes processes](https://mlsanity.com/paper/1501.00725/) (2015)
- [Causal Hidden Markov Model for Time Series Disease Forecasting](https://mlsanity.com/paper/2103.16391/) (2021)
- [BSODA: A Bipartite Scalable Framework for Online Disease Diagnosis](https://mlsanity.com/paper/2012.01065/) (2020)
- [Structural Hawkes Processes for Learning Causal Structure from Discrete-Time Event Sequences](https://mlsanity.com/paper/2305.05986/) (2023)
- [Local Causal Structure Learning in the Presence of Latent Variables](https://mlsanity.com/paper/2405.16225/) (2024)
- [Conditional Diffusion Models Based Conditional Independence Testing](https://mlsanity.com/paper/2412.11744/) (2024)
- [Harnessing Event Sensory Data for Error Pattern Prediction in Vehicles: A Language Model Approach](https://mlsanity.com/paper/2412.13041/) (2024)

## Abstract (arXiv)

Complex systems such as vehicles, patients, or genomes emit discrete event sequences whose operative question is causal, not predictive: which events cause which other events, and which cause higher-level outcomes such as failures or diseases? This question decomposes along two axes -- dependency type (event → event vs. event → outcome) and causal scope (single sequence vs. population) -- yielding four structurally distinct regimes with different identifiability conditions. No existing method addresses more than one, because all assume multi-stream structure with low vocabulary, and none scales beyond a few hundred event types. We present textscSeq2Cause, a unified framework that resolves all four regimes through a single shared primitive: a pretrained autoregressive model repurposed as an amortized conditional independence testing engine requiring no task-specific retraining. We establish a prediction--causality duality: the model's excess cross-entropy simultaneously bounds causal identification error across all four regimes, so that every improvement in next-token prediction tightens causal guarantees for free. On nonlinear SCMs (vocabularies up to 8,000 types) and real-world vehicle diagnostic logs (29K event types, 474 failure outcomes), textscSeq2Cause is the first method to populate all four regimes at scale with a single frozen backbone. Existing methods are either inapplicable, inaccurate, or computationally intractable in this setting.