# CausalBN-Bench: A Comprehensive Benchmark for Causal Learning Capability of LLMs

arXiv: https://arxiv.org/abs/2404.06349 · HTML: https://mlsanity.com/paper/2404.06349/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Apr 2024.

## Built on by (most cited first, 20 of 20)

- [Think or Not? Exploring Thinking Efficiency in Large Reasoning Models via an Information-Theoretic Lens](https://mlsanity.com/paper/2505.18237/) (2025)
- [Abduct, Act, Predict: Scaffolding Causal Inference for Automated Failure Attribution in Multi-Agent Systems](https://mlsanity.com/paper/2509.10401/) (2025)
- [CausalRAG: Integrating Causal Graphs into Retrieval-Augmented Generation](https://mlsanity.com/paper/2503.19878/) (2025)
- [Executable Counterfactuals: Improving LLMs' Causal Reasoning Through Code](https://mlsanity.com/paper/2510.01539/) (2025)
- [Ice Cream Doesn't Cause Drowning: Benchmarking LLMs Against Statistical Pitfalls in Causal Inference](https://mlsanity.com/paper/2505.13770/) (2025)
- [CausalReasoningBenchmark: A Real-World Benchmark for Disentangled Evaluation of Causal Identification and Estimation](https://mlsanity.com/paper/2602.20571/) (2026)
- [Multi-View Subgraph Neural Networks: Self-Supervised Learning with Scarce Labeled Data](https://mlsanity.com/paper/2404.12569/) (2024)
- [Can Large Language Models Infer Causal Relationships from Real-World Text?](https://mlsanity.com/paper/2505.18931/) (2025)
- [IGDA: Interactive Graph Discovery through Large Language Model Agents](https://mlsanity.com/paper/2502.17189/) (2025)
- [Revealing Multimodal Causality with Large Language Models](https://mlsanity.com/paper/2509.17784/) (2025)
- [A word association network methodology for evaluating implicit biases in LLMs compared to humans](https://mlsanity.com/paper/2510.24488/) (2025)
- [Caliper: Probing Lexical Anchors versus Causal Structure in LLMs](https://mlsanity.com/paper/2606.04915/) (2026)
- [CRAwDAD: Causal Reasoning Augmentation with Dual-Agent Debate](https://mlsanity.com/paper/2511.22854/) (2025)
- [Reasoning over Video: Evaluating How MLLMs Extract, Integrate, and Reconstruct Spatiotemporal Evidence](https://mlsanity.com/paper/2603.13091/) (2026)
- [CausalDS: Benchmarking Causal Reasoning in Data-Science Agents](https://mlsanity.com/paper/2607.08093/) (2026)
- [Eliciting associations between clinical variables from LLMs via comparison questions across populations](https://mlsanity.com/paper/2605.06335/) (2026)
- [Causal-Audit: Explicit and Auditable Graph-based Reasoning via Target-Aware Causal Chain Construction](https://mlsanity.com/paper/2607.15281/) (2026)
- [CausalVerify: End-to-End Verification of Causal Analyses by Language Models](https://mlsanity.com/paper/2609.07944/) (2026)
- [Evidence-Grounded Auditing of Identification Assumptions in Climate-Policy Causal Evaluations](https://mlsanity.com/paper/2609.30867/) (2026)
- [CCRV-Bench: Constraint-Based Evaluation of Causal Reasoning in Vision-Language Models](https://mlsanity.com/paper/2609.30979/) (2026)

## Abstract (arXiv)

The ability to understand causality significantly impacts the competence of large language models (LLMs) in output explanation and counterfactual reasoning, as causality reveals the underlying data distribution. However, the lack of a comprehensive benchmark currently limits the evaluation of LLMs' causal learning capabilities. To fill this gap, this paper develops CausalBN-Bench based on data from the causal research community, enabling comparative evaluations of LLMs against traditional causal learning algorithms. To provide a comprehensive investigation, we offer three tasks of varying difficulties, including correlation, causal skeleton, and causality identification. Evaluations of 19 leading LLMs reveal that, while closed-source LLMs show potential for simple causal relationships, they significantly lag behind traditional algorithms on larger-scale networks (>50 nodes). Specifically, LLMs struggle with collider structures but excel at chain structures, especially at long-chain causality analogous to Chain-of-Thought techniques. This supports the current prompt approaches while suggesting directions to enhance LLMs' causal reasoning capability. Furthermore, CausalBN-Bench incorporates background knowledge and training data into prompts to thoroughly unlock LLMs' text-comprehension ability during evaluation, whose findings indicate that, LLM understand causality through semantic associations with distinct entities, rather than directly from contextual information or numerical distributions.