mlsanityData to 7 Oct 2026

CausaLab: A Scalable Environment for Interactive Causal Discovery Toward AI Scientists

arXiv 2605.26029 ↗First posted 25 May 2026cs.AI cs.CLcited by 2

On the map: LLM agents and tool use → Medical and scientific domain agents

Abstract (arXiv)

We introduce CausaLab, a scalable environment for evaluating interactive causal discovery by LLM agents. Unlike prior evaluations, CausaLab evaluates both whether an agent can solve a problem using causal evidence and whether its answer is grounded in a faithful recovered causal mechanism. Each episode places an agent in a synthetic laboratory: it receives prior measurement records, intervenes on a manipulator crystal, and predicts the resonance frequency of a held-out reactor crystal governed by the same mechanism. The hidden data-generating process is a randomly sampled structural causal model (SCM), so success requires recovering both a causal graph and structural equations rather than recalling prior knowledge. Experiments show a persistent gap between prediction and mechanism recovery: in the purely observational 6-node setting, GPT-5.2-high reaches 92% task accuracy but only 0.471 all-edge F1. Mixed observation-intervention strategies improve structural fidelity, while pure intervention remains difficult even for strong agents. We identify premature stopping as a major weakness and show that consistency verification mitigates it. CausaLab therefore separates predictive success from causal understanding and exposes current LLM agents' limits as experimental causal reasoners.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Medical and scientific domain agents, one of 16 directions in LLM agents and tool use. Builds on 25 of its 61 references that are in the map. First posted on arXiv 25 May 2026.

Builds on

25 of 61 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023Passive learning of active causal strategies in agents and language models25 May

Other references · 24

  1. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  2. 2023Causal Reasoning and Large Language Models: Opening a New Frontier for Causalitycited by 348
  3. 2023Causal Parrots: Large Language Models May Talk Causality But Are Not Causalcited by 146
  4. 2023Can Large Language Models Infer Causation from Correlation?cited by 142
  5. 2023CLadder: Assessing Causal Reasoning in Language Modelscited by 102
  6. 2024DISCOVERYWORLD: A Virtual Environment for Developing and Evaluating Automated Scientific Discovery Agentscited by 89
  7. 2019Counterfactual Story Reasoning and Generationcited by 87
  8. 2023Can large language models build causal graphs?cited by 72
  9. 2022A Causal Framework to Quantify the Robustness of Mathematical Reasoning with Language Modelscited by 68
  10. 2024Efficient Causal Graph Discovery Using Large Language Modelscited by 51
  11. 2023Causal Order: The Key to Leveraging Imperfect Experts in Causal Inferencecited by 46
  12. 2015backShift: Learning causal cyclic graphs from unknown shift interventionscited by 37
12 more
  1. 2024Can Language Models Reason about Individualistic Human Values and Preferences?cited by 34
  2. 2025HypoBench: Towards Systematic and Principled Benchmarking for Hypothesis Generationcited by 34
  3. 2023CRAB: Assessing the Strength of Causal Relationships Between Real-world Eventscited by 27
  4. 2024Large Language Models are Effective Priors for Causal Graph Discoverycited by 23
  5. 2024Causal Evaluation of Language Modelscited by 18
  6. 2025Auto-Discovery-Bench: Diagnosing Structured State Tracking in Oracle-Guided Discoverycited by 15
  7. 2024CLEAR: Can Language Models Really Understand Causal Graphs?cited by 14
  8. 2023Preserving Commonsense Knowledge from Pre-trained Language Models via Causal Inferencecited by 13
  9. 2025Are Large Language Models Reliable AI Scientists? Assessing Reverse-Engineering of Black-Box Systemscited by 13
  10. 2024Teaching Transformers Causal Reasoning through Axiomatic Trainingcited by 12
  11. 2025DODO: Causal Structure Learning with Budgeted Interventionscited by 1
  12. 2026Linear-LLM-SCM: Benchmarking LLMs for Coefficient Elicitation in Linear-Gaussian Causal Modelscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Socratic agents for autonomous scientific discovery in high-dimensional physical systemscited by 2
  2. 2026CausalDS: Benchmarking Causal Reasoning in Data-Science Agentscited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2026CausalGame: Benchmarking Causal Thinking of LLM Agents in Games
  2. 2026CausalSteward: An Agentic Divide-Conquer-Combine Copilot for Causal Discovery
  3. 2026Why LLMs Fail at Causal Discovery and How Interventional Agents Escape
  4. 2026EvoSCM: Scientific Belief Revision Through Causal Model Evolution and Experimentation
  5. 2026Causal Retention in Interactive Agents: Interface Factorization and Selective Adaptation
  6. 2026Causal Ensemble Agent: Hierarchical Causal Discovery with LLM-guided Expert Reweighting
  7. 2026CausalArena: Benchmarking Causal Discovery in the Foundation Model Era
  8. 2026Separable Pathways for Causal Reasoning: How Architectural Scaffolding Enables Hypothesis-Space Restructuring in LLM Agents