mlsanityData to 14 Oct 2026

Sequential Causal Discovery with Noisy Language Model Priors

arXiv 2506.16234 ↗First posted 19 Jun 2025cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Causal discovery from observational data typically assumes access to complete data and availability of perfect domain experts. In practice, data often arrive in batches, are subject to sampling bias, and expert knowledge is scarce. Language Models (LMs) offer a surrogate for expert knowledge but suffer from hallucinations, inconsistencies, and bias. We present a hybrid framework that bridges these gaps by adaptively integrating sequential batch data with LM-derived noisy, expert knowledge while accounting for both data-induced and LM-induced biases. We propose a representation shift from Directed Acyclic Graph (DAG) to Partial Ancestral Graph (PAG), that accommodates ambiguities within a coherent framework, allowing grounding the global LM knowledge in local observational data. To guide LM interactions, we use a sequential optimization scheme that adaptively queries the most informative edges. Across varied datasets and LMs, we outperform prior work in structural accuracy and extend to parameter estimation, showing robustness to LM noise.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 34 of its 79 references that are in the map. First posted on arXiv 19 Jun 2025.

Builds on

34 of 79 references.

References · 34

  1. 2022Survey of Hallucination in Natural Language Generationcited by 2,075
  2. 2023Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generationcited by 819
  3. 2018DAGs with NO TEARS: Continuous Optimization for Structure Learningcited by 752
  4. 2023Causal Reasoning and Large Language Models: Opening a New Frontier for Causalitycited by 353
  5. 2013Causal Discovery with Continuous Additive Noise Modelscited by 255
  6. 2019Gradient-Based Neural DAG Learningcited by 227
  7. 2020Differentiable Causal Discovery from Interventional Datacited by 218
  8. 2019Learning Sparse Nonparametric DAGscited by 207
  9. 2013Learning Bayesian Networks: The Combination of Knowledge and Statistical Datacited by 187
  10. 2013Causal Inference and Causal Explanation with Background Knowledgecited by 170
  11. 2013Causal Inference in the Presence of Latent Variables and Selection Biascited by 166
  12. 2024Kernel Language Entropy: Fine-grained Uncertainty Quantification for LLMs from Semantic Similaritiescited by 159
22 more
  1. 2023Can Large Language Models Infer Causation from Correlation?cited by 146
  2. 2022DAGMA: Learning DAGs via M-matrices and a Log-Determinant Acyclicity Characterizationcited by 140
  3. 2017Permutation-based Causal Inference Algorithms with Interventionscited by 78
  4. 2023Can large language models build causal graphs?cited by 73
  5. 2022Causal Machine Learning for Healthcare and Precision Medicinecited by 68
  6. 2023Causal Discovery with Language Models as Imperfect Expertscited by 64
  7. 2019Causal Discovery with General Non-Linear Relationships Using Non-Linear ICAcited by 58
  8. 2021DAGs with No Curl: An Efficient DAG Structure Learning Approachcited by 53
  9. 2024Efficient Causal Graph Discovery Using Large Language Modelscited by 53
  10. 2023Causal Order: The Key to Leveraging Imperfect Experts in Causal Inferencecited by 48
  11. 2013Strong Completeness and Faithfulness in Bayesian Networkscited by 45
  12. 2021The impact of prior knowledge on causal structure learningcited by 35
  13. 2014Distinguishing cause from effect using observational data: methods and benchmarkscited by 33
  14. 2019Neuropathic Pain Diagnosis Simulator for Causal Discovery Algorithm Evaluationcited by 32
  15. 2024Integrating Large Language Models in Causal Discovery: A Statistical Causal Approachcited by 32
  16. 2022Can Foundation Models Talk Causality?cited by 30
  17. 2023Causal Structure Learning Supervised by Large Language Modelcited by 21
  18. 2023Subset verification and search algorithms for causal DAGscited by 12
  19. 2023Memory-Based Dual Gaussian Processes for Sequential Learningcited by 11
  20. 2023Expert-Aided Causal Discovery of Ancestral Graphscited by 8
  21. 2025Can LLMs Leverage Observational Data? Towards Data-Driven Causal Discovery with LLMscited by 5
  22. 2023Practical Batch Bayesian Sampling Algorithms for Online Adaptive Traffic Experimentationcited by 1

Built on by

Not cited yet.