mlsanityData to 7 Oct 2026

ExpliCa: Evaluating Explicit Causal Reasoning in Large Language Models

arXiv 2502.15487 ↗First posted 21 Feb 2025cs.CL cs.AIcited by 3

On the map: LLM reasoning → Causal reasoning in LLMs

Abstract (arXiv)

Large Language Models (LLMs) are increasingly used in tasks requiring interpretive and inferential accuracy. In this paper, we introduce ExpliCa, a new dataset for evaluating LLMs in explicit causal reasoning. ExpliCa uniquely integrates both causal and temporal relations presented in different linguistic orders and explicitly expressed by linguistic connectives. The dataset is enriched with crowdsourced human acceptability ratings. We tested LLMs on ExpliCa through prompting and perplexity-based metrics. We assessed seven commercial and open-source LLMs, revealing that even top models struggle to reach 0.80 accuracy. Interestingly, models tend to confound temporal relations with causal ones, and their performance is also strongly influenced by the linguistic order of the events. Finally, perplexity-based scores and prompting performance are differently affected by model size.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Causal reasoning in LLMs, one of 17 directions in LLM reasoning. Builds on 19 of its 50 references that are in the map. First posted on arXiv 21 Feb 2025.

Builds on

19 of 50 references.

Closest ancestors · 6

Same direction, newest first

  1. 2024CausalEval: Towards Better Causal Reasoning in Language Models22 Oct
  2. 2023Enhancing Causal Reasoning in Large Language Models: A Causal Attribution Model for Precision Fine-Tuning30 Dec
  3. 2023CLadder: Assessing Causal Reasoning in Language Models7 Dec
  4. 2023Is ChatGPT a Good Causal Reasoner? A Comprehensive Evaluation12 May
  5. 2023Causal Reasoning and Large Language Models: Opening a New Frontier for Causality28 Apr
  6. 2022e-CARE: a New Dataset for Exploring Explainable Causal Reasoning12 May

Other references · 13

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2024The Llama 3 Herd of Modelscited by 14,192
  3. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,928
  4. 2020Stanza: A Python Natural Language Processing Toolkit for Many Human Languagescited by 719
  5. 2023The Falcon Series of Open Language Modelscited by 526
  6. 2023Embers of Autoregression: Understanding Large Language Models Through the Problem They are Trained to Solvecited by 175
  7. 2023Causal Parrots: Large Language Models May Talk Causality But Are Not Causalcited by 146
  8. 2019Joint Reasoning for Temporal and Causal Relationscited by 60
  9. 2023CRAB: Assessing the Strength of Causal Relationships Between Real-world Eventscited by 27
  10. 2024Log Probabilities Are a Reliable Estimate of Semantic Plausibility in Base and Instruction-Tuned Language Modelscited by 24
  11. 2024Cause and Effect: Can Large Language Models Truly Understand Causality?cited by 23
  12. 2024Can You Learn Semantics Through Next-Word Prediction? The Case of Entailmentcited by 9
1 more
  1. 2021Predicting Directionality in Causal Relations in Textcited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 2.

3 papers.

  1. 2025Do Large Language Models Show Biases in Causal Learning? Insights from Contingency Judgmentcited by 1
  2. 2026InferenceEvolve: Towards Automated Causal Effect Estimators through Self-Evolving AIcited by 0
  3. 2026DeepImagine: Clinical Trial Outcome Prediction via Stepwise Local Counterfactual Imaginationscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024A Critical Review of Causal Reasoning Benchmarks for Large Language Models
  2. 2025Unveiling Causal Reasoning in Large Language Models: Reality or Mirage?
  3. 2024Large Language Models and Causal Inference in Collaboration: A Survey
  4. 2025CLEAR-3K: Assessing Causal Explanatory Capabilities in Language Models
  5. 2025A Survey on Enhancing Causal Reasoning Ability of Large Language Models
  6. 2026CausalFlip: A Benchmark for LLM Causal Judgment Beyond Semantic Matching
  7. 2026METER: Evaluating Multi-Level Contextual Causal Reasoning in Large Language Models
  8. 2024CausalBN-Bench: A Comprehensive Benchmark for Causal Learning Capability of LLMs