# Embracing the black box: Heading towards foundation models for causal discovery from time series data

arXiv: https://arxiv.org/abs/2402.09305 · HTML: https://mlsanity.com/paper/2402.09305/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 36 references that are in the map.
- First posted on arXiv 14 Feb 2024.

## Builds on (most cited first, 15 of 15)

- [Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting](https://mlsanity.com/paper/2012.07436/) (2020)
- [End to End Learning for Self-Driving Cars](https://mlsanity.com/paper/1604.07316/) (2016)
- [Benign Overfitting in Linear Regression](https://mlsanity.com/paper/1906.11300/) (2019)
- [Gradient-Based Neural DAG Learning](https://mlsanity.com/paper/1906.02226/) (2019)
- [Neural Autoregressive Flows](https://mlsanity.com/paper/1804.00779/) (2018)
- [Causal Recurrent Variational Autoencoder for Medical Time Series Generation](https://mlsanity.com/paper/2301.06574/) (2023)
- [Exploring Interpretable LSTM Neural Networks over Multi-Variable Data](https://mlsanity.com/paper/1905.12034/) (2019)
- [CASTLE: Regularization via Auxiliary Causal Graph Discovery](https://mlsanity.com/paper/2009.13180/) (2020)
- [Limits of End-to-End Learning](https://mlsanity.com/paper/1704.08305/) (2017)
- [Supervised Whole DAG Causal Discovery](https://mlsanity.com/paper/2006.04697/) (2020)
- [Causal discovery for observational sciences using supervised machine learning](https://mlsanity.com/paper/2202.12813/) (2022)
- [An Interpretable and Sparse Neural Network Model for Nonlinear Granger Causality Discovery](https://mlsanity.com/paper/1711.08160/) (2017)
- [Time Series Causal Link Estimation under Hidden Confounding using Knockoff Interventions](https://mlsanity.com/paper/2209.11497/) (2022)
- [seq2graph: Discovering Dynamic Dependencies from Multivariate Time Series with Multi-level Attention](https://mlsanity.com/paper/1812.04448/) (2018)
- [Estimating Granger Causality with Unobserved Confounders via Deep Latent-Variable Recurrent Neural Network](https://mlsanity.com/paper/1909.03704/) (2019)

## Built on by (most cited first, 7 of 7)

- [CausalRivers -- Scaling up benchmarking of causal discovery for real-world time-series](https://mlsanity.com/paper/2503.17452/) (2025)
- [Adversarial Causal Tuning for Realistic Time-series Generation](https://mlsanity.com/paper/2506.02084/) (2025)
- [FoundCause: Causal Discovery with Latent Confounders from Observational Data](https://mlsanity.com/paper/2606.17516/) (2026)
- [Large Causal Models for Temporal Causal Discovery](https://mlsanity.com/paper/2602.18662/) (2026)
- [TCD-Arena: Assessing Robustness of Time Series Causal Discovery Methods Against Assumption Violations](https://mlsanity.com/paper/2605.03045/) (2026)
- [Time Series Causal Discovery via Context-Conditioned and Causality-Augmented Pretraining](https://mlsanity.com/paper/2605.26759/) (2026)
- [One Thing Leads to Another: The Essentials of Time Series Causal Discovery](https://mlsanity.com/paper/2610.13430/) (2026)

## Abstract (arXiv)

Causal discovery from time series data encompasses many existing solutions, including those based on deep learning techniques. However, these methods typically do not endorse one of the most prevalent paradigms in deep learning: End-to-end learning. To address this gap, we explore what we call Causal Pretraining. A methodology that aims to learn a direct mapping from multivariate time series to the underlying causal graphs in a supervised manner. Our empirical findings suggest that causal discovery in a supervised manner is possible, assuming that the training and test time series samples share most of their dynamics. More importantly, we found evidence that the performance of Causal Pretraining can increase with data and model size, even if the additional data do not share the same dynamics. Further, we provide examples where causal discovery for real-world data with causally pretrained neural networks is possible within limits. We argue that this hints at the possibility of a foundation model for causal discovery.