# Data Generating Process to Evaluate Causal Discovery Techniques for Time Series Data

arXiv: https://arxiv.org/abs/2104.08043 · HTML: https://mlsanity.com/paper/2104.08043/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 40 references that are in the map.
- First posted on arXiv 16 Apr 2021.

## Builds on (most cited first, 12 of 12)

- [DAGs with NO TEARS: Continuous Optimization for Structure Learning](https://mlsanity.com/paper/1803.01422/) (2018)
- [DAG-GNN: DAG Structure Learning with Graph Neural Networks](https://mlsanity.com/paper/1904.10098/) (2019)
- [Detecting causal associations in large nonlinear time series datasets](https://mlsanity.com/paper/1702.07007/) (2017)
- [Learning Sparse Nonparametric DAGs](https://mlsanity.com/paper/1909.13189/) (2019)
- [DYNOTEARS: Structure Learning from Time-Series Data](https://mlsanity.com/paper/2002.00498/) (2020)
- [On the Role of Sparsity and DAG Constraints for Learning Linear DAGs](https://mlsanity.com/paper/2006.10201/) (2020)
- [CAM: Causal additive models, high-dimensional order search and penalized regression](https://mlsanity.com/paper/1310.1533/) (2013)
- [Causal Discovery from Heterogeneous/Nonstationary Data with Independent Changes](https://mlsanity.com/paper/1903.01672/) (2019)
- [Discovering contemporaneous and lagged causal relations in autocorrelated nonlinear time series datasets](https://mlsanity.com/paper/2003.03685/) (2020)
- [High-recall causal discovery for autocorrelated time series with latent confounders](https://mlsanity.com/paper/2007.01884/) (2020)
- [Causal structure learning from time series: Large regression coefficients may predict causal links better in practice than small p-values](https://mlsanity.com/paper/2002.09573/) (2020)
- [Learning DAGs without imposing acyclicity](https://mlsanity.com/paper/2006.03005/) (2020)

## Built on by (most cited first, 22 of 22)

- [CausalTime: Realistically Generated Time-series for Benchmarking of Causal Discovery](https://mlsanity.com/paper/2310.01753/) (2023)
- [A Survey on Causal Discovery Methods for I.I.D. and Time Series Data](https://mlsanity.com/paper/2303.15027/) (2023)
- [Signature Kernel Conditional Independence Tests in Causal Discovery for Stochastic Processes](https://mlsanity.com/paper/2402.18477/) (2024)
- [The Landscape of Causal Discovery Data: Grounding Causal Discovery in Real-World Applications](https://mlsanity.com/paper/2412.01953/) (2024)
- [CIPCaD-Bench: Continuous Industrial Process datasets for benchmarking Causal Discovery methods](https://mlsanity.com/paper/2208.01529/) (2022)
- [Unitless Unrestricted Markov-Consistent SCM Generation: Better Benchmark Datasets for Causal Discovery](https://mlsanity.com/paper/2503.17037/) (2025)
- [CausalDynamics: A large-scale benchmark for structural discovery of dynamical causal models](https://mlsanity.com/paper/2505.16620/) (2025)
- [Causal discovery in a complex industrial system: A time series benchmark](https://mlsanity.com/paper/2310.18654/) (2023)
- [Better Simulations for Validating Causal Discovery with the DAG-Adaptation of the Onion Method](https://mlsanity.com/paper/2405.13100/) (2024)
- [Boosting Synthetic Data Generation with Effective Nonlinear Causal Discovery](https://mlsanity.com/paper/2301.07427/) (2023)
- [CausalPlayground: Addressing Data-Generation Requirements in Cutting-Edge Causality Research](https://mlsanity.com/paper/2405.13092/) (2024)
- [Causal Inference in Non-linear Time-series using Deep Networks and Knockoff Counterfactuals](https://mlsanity.com/paper/2109.10817/) (2021)
- [CAnDOIT: Causal Discovery with Observational and Interventional Data from Time-Series](https://mlsanity.com/paper/2410.02844/) (2024)
- [OCDB: Revisiting Causal Discovery with a Comprehensive Benchmark and Evaluation Framework](https://mlsanity.com/paper/2406.04598/) (2024)
- [Causal Ordering for Structure Learning from Time Series](https://mlsanity.com/paper/2510.24639/) (2025)
- [Transformer Is Inherently a Causal Learner](https://mlsanity.com/paper/2601.05647/) (2026)
- [Large Causal Models for Temporal Causal Discovery](https://mlsanity.com/paper/2602.18662/) (2026)
- [GDBN: a Graph Neural Network Approach to Dynamic Bayesian Network](https://mlsanity.com/paper/2302.10804/) (2023)
- [Partially Specified Causal Simulations](https://mlsanity.com/paper/2309.10514/) (2023)
- [Learning Dynamic Bayesian Networks from Data: Foundations, First Principles and Numerical Comparisons](https://mlsanity.com/paper/2406.17585/) (2024)
- [Multidata Causal Discovery for Statistical Hurricane Intensity Forecasting](https://mlsanity.com/paper/2510.02050/) (2025)
- [Causal-Audit: A Framework for Risk Assessment of Assumption Violations in Time-Series Causal Discovery](https://mlsanity.com/paper/2604.02488/) (2026)

## Abstract (arXiv)

Going beyond correlations, the understanding and identification of causal relationships in observational time series, an important subfield of Causal Discovery, poses a major challenge. The lack of access to a well-defined ground truth for real-world data creates the need to rely on synthetic data for the evaluation of these methods. Existing benchmarks are limited in their scope, as they either are restricted to a "static" selection of data sets, or do not allow for a granular assessment of the methods' performance when commonly made assumptions are violated. We propose a flexible and simple to use framework for generating time series data, which is aimed at developing, evaluating, and benchmarking time series causal discovery methods. In particular, the framework can be used to fine tune novel methods on vast amounts of data, without "overfitting" them to a benchmark, but rather so they perform well in real-world use cases. Using our framework, we evaluate prominent time series causal discovery methods and demonstrate a notable degradation in performance when their assumptions are invalidated and their sensitivity to choice of hyperparameters. Finally, we propose future research directions and how our framework can support both researchers and practitioners.