# OCDB: Revisiting Causal Discovery with a Comprehensive Benchmark and Evaluation Framework

arXiv: https://arxiv.org/abs/2406.04598 · HTML: https://mlsanity.com/paper/2406.04598/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 54 references that are in the map.
- First posted on arXiv 7 Jun 2024.

## Builds on (most cited first, 30 of 31)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [DAGs with NO TEARS: Continuous Optimization for Structure Learning](https://mlsanity.com/paper/1803.01422/) (2018)
- [DAG-GNN: DAG Structure Learning with Graph Neural Networks](https://mlsanity.com/paper/1904.10098/) (2019)
- [Gradient-Based Neural DAG Learning](https://mlsanity.com/paper/1906.02226/) (2019)
- [Learning Sparse Nonparametric DAGs](https://mlsanity.com/paper/1909.13189/) (2019)
- [On the Role of Sparsity and DAG Constraints for Learning Linear DAGs](https://mlsanity.com/paper/2006.10201/) (2020)
- [Graphite: Iterative Generative Modeling of Graphs](https://mlsanity.com/paper/1803.10459/) (2018)
- [Score matching enables causal discovery of nonlinear additive noise models](https://mlsanity.com/paper/2203.04413/) (2022)
- [Rethinking Interpretability in the Era of Large Language Models](https://mlsanity.com/paper/2402.01761/) (2024)
- [Causal-learn: Causal Discovery in Python](https://mlsanity.com/paper/2307.16405/) (2023)
- [Deep End-to-end Causal Inference](https://mlsanity.com/paper/2202.02195/) (2022)
- [Causal Interpretability for Machine Learning -- Problems, Methods and Evaluation](https://mlsanity.com/paper/2003.03934/) (2020)
- [BCD Nets: Scalable Variational Approaches for Bayesian Causal Discovery](https://mlsanity.com/paper/2112.02761/) (2021)
- [DAGs with No Fears: A Closer Look at Continuous Optimization for Learning Bayesian Networks](https://mlsanity.com/paper/2010.09133/) (2020)
- [gCastle: A Python Toolbox for Causal Discovery](https://mlsanity.com/paper/2111.15155/) (2021)
- [Causal Discovery Toolbox: Uncover causal relationships in Python](https://mlsanity.com/paper/1903.02278/) (2019)
- [Interpretable Models for Granger Causality Using Self-explaining Neural Networks](https://mlsanity.com/paper/2101.07600/) (2021)
- [CUTS+: High-dimensional Causal Discovery from Irregular Time-series](https://mlsanity.com/paper/2305.05890/) (2023)
- [CausalTime: Realistically Generated Time-series for Benchmarking of Causal Discovery](https://mlsanity.com/paper/2310.01753/) (2023)
- [Cause and Effect: Can Large Language Models Truly Understand Causality?](https://mlsanity.com/paper/2402.18139/) (2024)
- [Data Generating Process to Evaluate Causal Discovery Techniques for Time Series Data](https://mlsanity.com/paper/2104.08043/) (2021)
- [Differentiable Causal Discovery Under Latent Interventions](https://mlsanity.com/paper/2203.02336/) (2022)
- [THP: Topological Hawkes Processes for Learning Causal Structure on Event Sequences](https://mlsanity.com/paper/2105.10884/) (2021)
- [Causal Discovery with Latent Confounders Based on Higher-Order Cumulants](https://mlsanity.com/paper/2305.19582/) (2023)
- [Structural Hawkes Processes for Learning Causal Structure from Discrete-Time Event Sequences](https://mlsanity.com/paper/2305.05986/) (2023)
- [Sparsity-Guided Holistic Explanation for LLMs with Interpretable Inference-Time Intervention](https://mlsanity.com/paper/2312.15033/) (2023)
- [A Meta-Reinforcement Learning Algorithm for Causal Discovery](https://mlsanity.com/paper/2207.08457/) (2022)
- [Context-Specific Causal Discovery for Categorical Data Using Staged Trees](https://mlsanity.com/paper/2106.04416/) (2021)
- [CIPCaD-Bench: Continuous Industrial Process datasets for benchmarking Causal Discovery methods](https://mlsanity.com/paper/2208.01529/) (2022)
- [From Temporal to Contemporaneous Iterative Causal Discovery in the Presence of Latent Confounders](https://mlsanity.com/paper/2306.00624/) (2023)

## Built on by (most cited first, 2 of 2)

- [CausalRivers -- Scaling up benchmarking of causal discovery for real-world time-series](https://mlsanity.com/paper/2503.17452/) (2025)
- [TCD-Arena: Assessing Robustness of Time Series Causal Discovery Methods Against Assumption Violations](https://mlsanity.com/paper/2605.03045/) (2026)

## Abstract (arXiv)

Large language models (LLMs) have excelled in various natural language processing tasks, but challenges in interpretability and trustworthiness persist, limiting their use in high-stakes fields. Causal discovery offers a promising approach to improve transparency and reliability. However, current evaluations are often one-sided and lack assessments focused on interpretability performance. Additionally, these evaluations rely on synthetic data and lack comprehensive assessments of real-world datasets. These lead to promising methods potentially being overlooked. To address these issues, we propose a flexible evaluation framework with metrics for evaluating differences in causal structures and causal effects, which are crucial attributes that help improve the interpretability of LLMs. We introduce the Open Causal Discovery Benchmark (OCDB), based on real data, to promote fair comparisons and drive optimization of algorithms. Additionally, our new metrics account for undirected edges, enabling fair comparisons between Directed Acyclic Graphs (DAGs) and Completed Partially Directed Acyclic Graphs (CPDAGs). Experimental results show significant shortcomings in existing algorithms' generalization capabilities on real data, highlighting the potential for performance improvement and the importance of our framework in advancing causal discovery techniques.