mlsanityData to 7 Oct 2026

InterveneBench: Benchmarking LLMs for Intervention Reasoning and Causal Study Design in Real Social Systems

arXiv 2603.15542 ↗First posted 16 Mar 2026cs.CY cs.AIcited by 3

On the map: LLM reasoning → Causal reasoning in LLMs

Abstract (arXiv)

Causal inference in social science relies on end-to-end, intervention-centered research-design reasoning grounded in real-world policy interventions, but current benchmarks fail to evaluate this capability of large language models (LLMs). We present InterveneBench, a benchmark designed to assess such reasoning in realistic social settings. Each instance in InterveneBench is derived from an empirical social science study and requires models to reason about policy interventions and identification assumptions without access to predefined causal graphs or structural equations. InterveneBench comprises 744 peer-reviewed studies across diverse policy domains. Experimental results show that state-of-the-art LLMs struggle under this setting. To address this limitation, we further propose a multi-agent framework, STRIDES. It achieves significant performance improvements over state-of-the-art reasoning models. Our code and data are available at https://github.com/Sii-yuning/STRIDES.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Causal reasoning in LLMs, one of 17 directions in LLM reasoning. Builds on 10 of its 47 references that are in the map. First posted on arXiv 16 Mar 2026.

Builds on

10 of 47 references.

References · 10

  1. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  2. 2025Qwen3 Technical Reportcited by 8,195
  3. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  4. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
  5. 2024LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Modelscited by 1,442
  6. 2025gpt-oss-120b & gpt-oss-20b Model Cardcited by 1,258
  7. 2025DeepSeek-V3.2: Pushing the Frontier of Open Large Language Modelscited by 755
  8. 2025Kimi K2: Open Agentic Intelligencecited by 523
  9. 2025ResearchBench: Benchmarking LLMs in Scientific Discovery via Inspiration-Based Task Decompositioncited by 57
  10. 2025DocLens : A Tool-Augmented Multi-Agent Framework for Long Visual Document Understandingcited by 15

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026CausalDS: Benchmarking Causal Reasoning in Data-Science Agentscited by 1
  2. 2026CausalVerify: End-to-End Verification of Causal Analyses by Language Modelscited by 0
  3. 2026CCRV-Bench: Constraint-Based Evaluation of Causal Reasoning in Vision-Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Evaluating Interventional Reasoning Capabilities of Large Language Models
  2. 2024A Critical Review of Causal Reasoning Benchmarks for Large Language Models
  3. 2025Ice Cream Doesn't Cause Drowning: Benchmarking LLMs Against Statistical Pitfalls in Causal Inference
  4. 2025Causal MAS: A Survey of Large Language Model Architectures for Discovery and Effect Estimation
  5. 2023Causal Reasoning and Large Language Models: Opening a New Frontier for Causality
  6. 2023Towards CausalGPT: A Multi-Agent Approach for Faithful Knowledge Reasoning via Promoting Causal Consistency in LLMs
  7. 2025Exploring Large Language Model Agents for Piloting Social Experiments
  8. 2024Sense and Sensitivity: Evaluating the simulation of social dynamics via Large Language Models