# CausaLab: A Scalable Environment for Interactive Causal Discovery Toward AI Scientists

arXiv: https://arxiv.org/abs/2605.26029 · HTML: https://mlsanity.com/paper/2605.26029/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Medical and scientific domain agents, one of 16 directions in LLM agents and tool use.
- Builds on 25 of its 61 references that are in the map.
- First posted on arXiv 25 May 2026.

## Placed in

- LLM agents and tool use / [Medical and scientific domain agents](https://mlsanity.com/agents/medical-and-scientific-domain-agents/)

## Builds on (most cited first, 25 of 25)

- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [Causal Reasoning and Large Language Models: Opening a New Frontier for Causality](https://mlsanity.com/paper/2305.00050/) (2023)
- [Causal Parrots: Large Language Models May Talk Causality But Are Not Causal](https://mlsanity.com/paper/2308.13067/) (2023)
- [Can Large Language Models Infer Causation from Correlation?](https://mlsanity.com/paper/2306.05836/) (2023)
- [CLadder: Assessing Causal Reasoning in Language Models](https://mlsanity.com/paper/2312.04350/) (2023)
- [DISCOVERYWORLD: A Virtual Environment for Developing and Evaluating Automated Scientific Discovery Agents](https://mlsanity.com/paper/2406.06769/) (2024)
- [Counterfactual Story Reasoning and Generation](https://mlsanity.com/paper/1909.04076/) (2019)
- [Can large language models build causal graphs?](https://mlsanity.com/paper/2303.05279/) (2023)
- [A Causal Framework to Quantify the Robustness of Mathematical Reasoning with Language Models](https://mlsanity.com/paper/2210.12023/) (2022)
- [Efficient Causal Graph Discovery Using Large Language Models](https://mlsanity.com/paper/2402.01207/) (2024)
- [Causal Order: The Key to Leveraging Imperfect Experts in Causal Inference](https://mlsanity.com/paper/2310.15117/) (2023)
- [backShift: Learning causal cyclic graphs from unknown shift interventions](https://mlsanity.com/paper/1506.02494/) (2015)
- [HypoBench: Towards Systematic and Principled Benchmarking for Hypothesis Generation](https://mlsanity.com/paper/2504.11524/) (2025)
- [Can Language Models Reason about Individualistic Human Values and Preferences?](https://mlsanity.com/paper/2410.03868/) (2024)
- [CRAB: Assessing the Strength of Causal Relationships Between Real-world Events](https://mlsanity.com/paper/2311.04284/) (2023)
- [Large Language Models are Effective Priors for Causal Graph Discovery](https://mlsanity.com/paper/2405.13551/) (2024)
- [Passive learning of active causal strategies in agents and language models](https://mlsanity.com/paper/2305.16183/) (2023)
- [Causal Evaluation of Language Models](https://mlsanity.com/paper/2405.00622/) (2024)
- [Auto-Discovery-Bench: Diagnosing Structured State Tracking in Oracle-Guided Discovery](https://mlsanity.com/paper/2502.15224/) (2025)
- [CLEAR: Can Language Models Really Understand Causal Graphs?](https://mlsanity.com/paper/2406.16605/) (2024)
- [Are Large Language Models Reliable AI Scientists? Assessing Reverse-Engineering of Black-Box Systems](https://mlsanity.com/paper/2505.17968/) (2025)
- [Preserving Commonsense Knowledge from Pre-trained Language Models via Causal Inference](https://mlsanity.com/paper/2306.10790/) (2023)
- [Teaching Transformers Causal Reasoning through Axiomatic Training](https://mlsanity.com/paper/2407.07612/) (2024)
- [DODO: Causal Structure Learning with Budgeted Interventions](https://mlsanity.com/paper/2510.08207/) (2025)
- [Linear-LLM-SCM: Benchmarking LLMs for Coefficient Elicitation in Linear-Gaussian Causal Models](https://mlsanity.com/paper/2602.10282/) (2026)

## Built on by (most cited first, 4 of 4)

- [Socratic agents for autonomous scientific discovery in high-dimensional physical systems](https://mlsanity.com/paper/2606.26722/) (2026)
- [CausalDS: Benchmarking Causal Reasoning in Data-Science Agents](https://mlsanity.com/paper/2607.08093/) (2026)
- [ULTRADISCOVERY: Abductive Exploration in an Interconnected, Epistemically Open Universe](https://mlsanity.com/paper/2610.03092/) (2026)
- [CivWorldBench: Evaluating Language World Models in Structured Multi-Agent Sandbox Worlds](https://mlsanity.com/paper/2610.16450/) (2026)

## Abstract (arXiv)

We introduce CausaLab, a scalable environment for evaluating interactive causal discovery by LLM agents. Unlike prior evaluations, CausaLab evaluates both whether an agent can solve a problem using causal evidence and whether its answer is grounded in a faithful recovered causal mechanism. Each episode places an agent in a synthetic laboratory: it receives prior measurement records, intervenes on a manipulator crystal, and predicts the resonance frequency of a held-out reactor crystal governed by the same mechanism. The hidden data-generating process is a randomly sampled structural causal model (SCM), so success requires recovering both a causal graph and structural equations rather than recalling prior knowledge. Experiments show a persistent gap between prediction and mechanism recovery: in the purely observational 6-node setting, GPT-5.2-high reaches 92% task accuracy but only 0.471 all-edge F1. Mixed observation-intervention strategies improve structural fidelity, while pure intervention remains difficult even for strong agents. We identify premature stopping as a major weakness and show that consistency verification mitigates it. CausaLab therefore separates predictive success from causal understanding and exposes current LLM agents' limits as experimental causal reasoners.