mlsanityData to 14 Oct 2026

Active Causal Discovery Benchmark: Evaluating LLM Agents Under Budgeted Interventions

arXiv 2609.31675 ↗First posted 16 Sep 2026cs.LG cs.AIcited by 1

Not on the map. Why?

Abstract (arXiv)

We introduce the Active Causal Discovery Benchmark (ACDB), an SCM-grounded environment for evaluating whether LLM agents recover causal graph structure from observations and budget-constrained hard interventions. ACDB pairs a linear-Gaussian world generator with a fixed observe-intervene-submit API and a three-layer scoring contract that separates skeleton recovery, DAG recovery, and intervention efficiency. On the current six-level ladder, PC with a greedy active orientation heuristic is the strongest non-oracle method (directed F1 42.7%, SHD 4.79), ahead of Claude Sonnet 4.6 raw active (31.7%, 7.25) and GPT-5.4 raw active (22.9%, 9.27). The most informative diagnostic is the precision-recall decomposition: PC under-commits with high precision, LLMs over-commit with lower precision, and statistical-tool access often increases abstention rather than useful intervention. A structure-blind random DAG baseline reaches 23.6% directed F1 on this dense v0 ladder; a density probe lowers this floor to 16.9%, motivating the v1 calibration pass. The current results should therefore be read as a benchmark audit and calibration report, not as evidence that current LLMs solve active causal discovery.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 9 of its 15 references that are in the map. First posted on arXiv 16 Sep 2026.

Builds on

9 of 15 references.

References · 9

  1. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,373
  2. 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 2,006
  3. 2023AgentBench: Evaluating LLMs as Agentscited by 1,267
  4. 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environmentscited by 1,195
  5. 2023GAIA: a benchmark for General AI Assistantscited by 1,128
  6. 2023Causal Reasoning and Large Language Models: Opening a New Frontier for Causalitycited by 353
  7. 2023Causal Parrots: Large Language Models May Talk Causality But Are Not Causalcited by 150
  8. 2023CLadder: Assessing Causal Reasoning in Language Modelscited by 107
  9. 2023Can large language models build causal graphs?cited by 73

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.