mlsanityData to 14 Oct 2026

Causal Order: The Key to Leveraging Imperfect Experts in Causal Inference

arXiv 2310.15117 ↗First posted 23 Oct 2023cs.AI cs.CLcited by 48

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) have been used as experts to infer causal graphs, often by repeatedly applying a pairwise prompt that asks about the causal relationship of each variable pair. However, such experts, including human domain experts, cannot distinguish between direct and indirect effects given a pairwise prompt. Therefore, instead of the graph, we propose that causal order be used as a more stable output interface for utilizing expert knowledge. Even when querying a perfect expert with a pairwise prompt, we show that the inferred graph can have significant errors whereas the causal order is always correct. In practice, however, LLMs are imperfect experts and we find that pairwise prompts lead to multiple cycles. Hence, we propose the triplet method, a novel querying strategy that introduces an auxiliary variable for every variable pair and instructs the LLM to avoid cycles within this triplet. It then uses a voting-based ensemble method that results in higher accuracy and fewer cycles while ensuring cost efficiency. Across multiple real-world graphs, such a triplet-based method yields a more accurate order than the pairwise prompt, using both LLMs and human annotators. The triplet method enhances robustness by repeatedly querying an expert with different auxiliary variables, enabling smaller models like Phi-3 and Llama-3 8B Instruct to surpass GPT-4 with pairwise prompting. For practical usage, we show how the expert-provided causal order from the triplet method can be used to reduce error in downstream graph discovery and effect inference tasks.

Key facts. Cited by 48 papers in the map (data through 14 Oct 2026). Builds on 16 of its 41 references that are in the map. First posted on arXiv 23 Oct 2023.

Builds on

16 of 41 references.

References · 16

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  3. 2018DAGs with NO TEARS: Continuous Optimization for Structure Learningcited by 752
  4. 2023Causal Reasoning and Large Language Models: Opening a New Frontier for Causalitycited by 353
  5. 2013Causal Inference and Causal Explanation with Background Knowledgecited by 170
  6. 2022Score matching enables causal discovery of nonlinear additive noise modelscited by 111
  7. 2020DoWhy: An End-to-End Library for Causal Inferencecited by 95
  8. 2023Can large language models build causal graphs?cited by 73
  9. 2019Causal structure based root cause analysis of outlierscited by 65
  10. 2023Causal Discovery with Language Models as Imperfect Expertscited by 64
  11. 2022LMPriors: Pre-Trained Language Models as Task-Specific Priorscited by 56
  12. 2023Integrating Large Language Model for Improved Causal Discoverycited by 54
4 more
  1. 2021The impact of prior knowledge on causal structure learningcited by 35
  2. 2019Neuropathic Pain Diagnosis Simulator for Causal Discovery Algorithm Evaluationcited by 32
  3. 2013Learning Bayesian Networks: A Unification for Discrete and Gaussian Domainscited by 21
  4. 2021Incorporating Causal Graphical Prior Knowledge into Predictive Modeling via Simple Data Augmentationcited by 10

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 22; 2025: 11; 2026: 14.

48 papers.

Sort:

By area:

  1. 2024Mitigating Modality Prior-Induced Hallucinations in Multimodal Large Language Models via Deciphering Attention Causalitycited by 42
  2. 2024Integrating Large Language Models in Causal Discovery: A Statistical Causal Approachcited by 32
  3. 2024Large Language Models and Causal Inference in Collaboration: A Surveycited by 26
  4. 2024Causality for Large Language Modelscited by 25
  5. 2024Large Language Models are Effective Priors for Causal Graph Discoverycited by 24
  6. 2024MECD: Unlocking Multi-Event Causal Discovery in Video Reasoningcited by 23
  7. 2023Causal Structure Learning Supervised by Large Language Modelcited by 21
  8. 2024Causal Evaluation of Language Modelscited by 18
  9. 2024End-To-End Causal Effect Estimation from Unstructured Natural Language Datacited by 18
  10. 2024Causal Graph Discovery with Retrieval-Augmented Generation based Large Language Modelscited by 16
  11. 2024Large Language Models for Constrained-Based Causal Discoverycited by 15
  12. 2024RealTCD: Temporal Causal Discovery from Interventional Data with Large Language Modelcited by 13
36 more
  1. 2024CausalGraph2LLM: Evaluating LLMs for Causal Queriescited by 12
  2. 2024Discovering and Reasoning of Causality in the Hidden World with Large Language Modelscited by 10
  3. 2025Executable Counterfactuals: Improving LLMs' Causal Reasoning Through Codecited by 10
  4. 2025A Survey on Enhancing Causal Reasoning Ability of Large Language Modelscited by 8
  5. 2025LLM Cannot Discover Causality, and Should Be Restricted to Non-Decisional Support in Causal Discoverycited by 8
  6. 2024LLM4GRN: Discovering Causal Gene Regulatory Networks with LLMs -- Evaluation through Synthetic Data Generationcited by 7
  7. 2024Prompting Strategies for Enabling Large Language Models to Infer Causation from Correlationcited by 7
  8. 2024Evaluating Interventional Reasoning Capabilities of Large Language Modelscited by 6
  9. 2024LLM-initialized Differentiable Causal Discoverycited by 6
  10. 2024Multimodal Causal Reasoning Benchmark: Challenging Vision Large Language Models to Discern Causal Links Across Modalitiescited by 5
  11. 2025Fairness-Driven LLM-based Causal Discovery with Active Learning and Dynamic Scoringcited by 5
  12. 2026CausaLab: A Scalable Environment for Interactive Causal Discovery Toward AI Scientistscited by 4
  13. 2025IGDA: Interactive Graph Discovery through Large Language Model Agentscited by 3
  14. 2025Revealing Multimodal Causality with Large Language Modelscited by 3
  15. 2026CauScientist: Teaching LLMs to Respect Data for Causal Discoverycited by 3
  16. 2024Large Language Models as Co-Pilots for Causal Inference in Medical Studiescited by 2
  17. 2024MedG-KRP: Medical Graph Knowledge Representation Probingcited by 2
  18. 2025Score-informed Neural Operator for Enhancing Ordering-based Causal Discoverycited by 2
  19. 2025Evaluation of Causal Reasoning for Large Language Models in Contextualized Clinical Scenarios of Laboratory Test Interpretationcited by 2
  20. 2026Dynamic Expert-Guided Model Averaging for Causal Discoverycited by 2
  21. 2025Improving constraint-based discovery with robust propagation and reliable LLM priorscited by 1
  22. 2026Leveraging Large Language Models for Causal Discovery: a Constraint-based, Argumentation-driven Approachcited by 1
  23. 2026Consistency evaluation of benchmarks used for causal discoverycited by 1
  24. 2026CauTion: Knowing When to Trust LLMs for Ensemble Causal Discoverycited by 1
  25. 2024Membership Testing in Markov Equivalence Classes via Independence Query Oraclescited by 0
  26. 2024Retrieving Classes of Causal Orders with Inconsistent Knowledge Basescited by 0
  27. 2025Sequential Causal Discovery with Noisy Language Model Priorscited by 0
  28. 2025LLM-based Agents for Automated Confounder Discovery and Subgroup Analysis in Causal Inferencecited by 0
  29. 2026DMCD: Semantic-Statistical Framework for Causal Discoverycited by 0
  30. 2026Causal Discovery Should Embrace the Wisdom of the Crowdcited by 0
  31. 2026Enhancing Regime Shift Detection Using Unstructured Data: A Study on the Treasury Marketcited by 0
  32. 2026GENESIS: Towards Explainable Causal Discoverycited by 0
  33. 2026LLM-Augmented Causal Discovery: Probabilistic Fusion of Edge Existence and Orientationcited by 0
  34. 2026seq2cause: One Autoregressive Backbone, Four Causal Discovery Tasks in Event Sequencescited by 0
  35. 2026Overcoming Challenges of Interpretive Structural Modeling with Large Language Modelscited by 0
  36. 2026Coupling Noisy Pairwise Knowledge to the DAG Posterior for Causal Discoverycited by 0