mlsanityData to 7 Oct 2026

OCDB: Revisiting Causal Discovery with a Comprehensive Benchmark and Evaluation Framework

arXiv 2406.04598 ↗First posted 7 Jun 2024cs.AIcited by 1

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) have excelled in various natural language processing tasks, but challenges in interpretability and trustworthiness persist, limiting their use in high-stakes fields. Causal discovery offers a promising approach to improve transparency and reliability. However, current evaluations are often one-sided and lack assessments focused on interpretability performance. Additionally, these evaluations rely on synthetic data and lack comprehensive assessments of real-world datasets. These lead to promising methods potentially being overlooked. To address these issues, we propose a flexible evaluation framework with metrics for evaluating differences in causal structures and causal effects, which are crucial attributes that help improve the interpretability of LLMs. We introduce the Open Causal Discovery Benchmark (OCDB), based on real data, to promote fair comparisons and drive optimization of algorithms. Additionally, our new metrics account for undirected edges, enabling fair comparisons between Directed Acyclic Graphs (DAGs) and Completed Partially Directed Acyclic Graphs (CPDAGs). Experimental results show significant shortcomings in existing algorithms' generalization capabilities on real data, highlighting the potential for performance improvement and the importance of our framework in advancing causal discovery techniques.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 31 of its 54 references that are in the map. First posted on arXiv 7 Jun 2024.

Builds on

31 of 54 references.

References · 31

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2018DAGs with NO TEARS: Continuous Optimization for Structure Learningcited by 745
  3. 2019DAG-GNN: DAG Structure Learning with Graph Neural Networkscited by 329
  4. 2019Gradient-Based Neural DAG Learningcited by 227
  5. 2019Learning Sparse Nonparametric DAGscited by 207
  6. 2020On the Role of Sparsity and DAG Constraints for Learning Linear DAGscited by 186
  7. 2018Graphite: Iterative Generative Modeling of Graphscited by 112
  8. 2024Rethinking Interpretability in the Era of Large Language Modelscited by 110
  9. 2022Score matching enables causal discovery of nonlinear additive noise modelscited by 108
  10. 2023Causal-learn: Causal Discovery in Pythoncited by 108
  11. 2022Deep End-to-end Causal Inferencecited by 93
  12. 2020Causal Interpretability for Machine Learning -- Problems, Methods and Evaluationcited by 82
19 more
  1. 2021BCD Nets: Scalable Variational Approaches for Bayesian Causal Discoverycited by 68
  2. 2020DAGs with No Fears: A Closer Look at Continuous Optimization for Learning Bayesian Networkscited by 66
  3. 2021gCastle: A Python Toolbox for Causal Discoverycited by 60
  4. 2019Causal Discovery Toolbox: Uncover causal relationships in Pythoncited by 49
  5. 2021Interpretable Models for Granger Causality Using Self-explaining Neural Networkscited by 44
  6. 2023CUTS+: High-dimensional Causal Discovery from Irregular Time-seriescited by 39
  7. 2023CausalTime: Realistically Generated Time-series for Benchmarking of Causal Discoverycited by 35
  8. 2024Cause and Effect: Can Large Language Models Truly Understand Causality?cited by 23
  9. 2021Data Generating Process to Evaluate Causal Discovery Techniques for Time Series Datacited by 22
  10. 2022Differentiable Causal Discovery Under Latent Interventionscited by 21
  11. 2021THP: Topological Hawkes Processes for Learning Causal Structure on Event Sequencescited by 19
  12. 2023Causal Discovery with Latent Confounders Based on Higher-Order Cumulantscited by 15
  13. 2023Structural Hawkes Processes for Learning Causal Structure from Discrete-Time Event Sequencescited by 14
  14. 2023Sparsity-Guided Holistic Explanation for LLMs with Interpretable Inference-Time Interventioncited by 12
  15. 2022A Meta-Reinforcement Learning Algorithm for Causal Discoverycited by 11
  16. 2021Context-Specific Causal Discovery for Categorical Data Using Staged Treescited by 10
  17. 2022CIPCaD-Bench: Continuous Industrial Process datasets for benchmarking Causal Discovery methodscited by 10
  18. 2023From Temporal to Contemporaneous Iterative Causal Discovery in the Presence of Latent Confounderscited by 9
  19. 2024Understanding Large Language Model Behaviors through Interactive Counterfactual Generation and Analysiscited by 8

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025CausalRivers -- Scaling up benchmarking of causal discovery for real-world time-seriescited by 24