mlsanityData to 14 Oct 2026

CausalARC: Abstract Reasoning with Causal World Models

arXiv 2509.03636 ↗First posted 3 Sep 2025cs.AI cs.CL cs.LGcited by 4

On the map: World models → World models for language agents

also in 1 more
Abstract (arXiv)

On-the-fly reasoning often requires adaptation to novel problems under limited data and distribution shift. This work introduces CausalARC: an experimental testbed for AI reasoning in low-data and out-of-distribution regimes, modeled after the Abstraction and Reasoning Corpus (ARC). Each CausalARC reasoning task is sampled from a fully specified causal world model, formally expressed as a structural causal model. Principled data augmentations provide observational, interventional, and counterfactual feedback about the world model in the form of few-shot, in-context learning demonstrations. As a proof-of-concept, we illustrate the use of CausalARC for four language model evaluation settings: (1) abstract reasoning with test-time training, (2) counterfactual reasoning with in-context learning, (3) program synthesis, and (4) causal discovery with logical reasoning. Within- and between-model performance varied heavily across tasks, indicating room for significant improvement in language model reasoning.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in World models for language agents, one of 11 directions in World models. Builds on 29 of its 49 references that are in the map. First posted on arXiv 3 Sep 2025.

Builds on

29 of 49 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024Robust agents learn causal world models16 Feb
  2. 2023Reasoning with Language Model is Planning with World Model24 May
  3. 2020Language Models are Few-Shot Learners28 May

Other references · 26

  1. 2018World Modelscited by 1,513
  2. 2024GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in Large Language Modelscited by 550
  3. 2022Efficient Test-Time Model Adaptation without Forgettingcited by 469
  4. 2023Larger language models do in-context learning differentlycited by 394
  5. 2023A Comprehensive Survey on Test-Time Adaptation under Distribution Shiftscited by 359
  6. 2025The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexitycited by 302
  7. 2023Causal Parrots: Large Language Models May Talk Causality But Are Not Causalcited by 150
  8. 2023Can Large Language Models Infer Causation from Correlation?cited by 146
  9. 2023CLadder: Assessing Causal Reasoning in Language Modelscited by 107
  10. 2024ARC Prize 2024: Technical Reportcited by 104
  11. 2023The ConceptARC Benchmark: Evaluating Understanding and Generalization in the ARC Domaincited by 101
  12. 2024The Surprising Effectiveness of Test-Time Training for Few-Shot Learningcited by 96
14 more
  1. 2013Probabilities of Causation: Bounds and Identificationcited by 90
  2. 2023Is ChatGPT a Good Causal Reasoner? A Comprehensive Evaluationcited by 69
  3. 2024Combining Induction and Transduction for Abstract Reasoningcited by 53
  4. 2019Human few-shot learning of compositional instructionscited by 47
  5. 2023Causal-Discovery Performance of ChatGPT in the context of Neuropathic Pain Diagnosiscited by 36
  6. 2024Addressing the Abstraction and Reasoning Corpus via Procedural Example Generationcited by 33
  7. 2024H-ARC: A Robust Estimate of Human Performance on the Abstraction and Reasoning Corpus Benchmarkcited by 33
  8. 2024Reasoning Elicitation in Language Models via Counterfactual Feedbackcited by 14
  9. 2025PhantomWiki: On-Demand Datasets for Reasoning and Retrieval Evaluationcited by 14
  10. 2025RE-IMAGINE: Symbolic Benchmark Synthesis for Reasoning Evaluationcited by 11
  11. 2020A Weaker Faithfulness Assumption based on Triple Interactionscited by 10
  12. 2024Does Reasoning Emerge? Examining the Probabilities of Causation in Large Language Modelscited by 8
  13. 2025Compositional Causal Reasoning Evaluation in Language Modelscited by 6
  14. 2025Since Faithfulness Fails: The Performance Limits of Neural Causal Discoverycited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 4.

4 papers.

  1. 2026Implicit Rule Induction with Test-Time Task Embeddings in ARC-like Taskscited by 2
  2. 2026CauSim: Scaling Causal Reasoning with Increasingly Complex Causal Simulatorscited by 0
  3. 2026A Taxonomy of Cognitive Capability Gaps in Generative and Agentic AIcited by 0
  4. 2026Position: Reasoning is a Learnable Rule-Based Processcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025TempoBench: Reasoning Execution Without Causal Attribution Is Just Simulation
  2. 2026Better Think Thrice: Learning to Reason Causally with Double Counterfactual Consistency
  3. 2024A Critical Review of Causal Reasoning Benchmarks for Large Language Models
  4. 2025Unveiling Causal Reasoning in Large Language Models: Reality or Mirage?
  5. 2025Causal Reasoning in Pieces: Modular In-Context Learning for Causal Discovery
  6. 2024Language Agents Meet Causality -- Bridging LLMs and Causal World Models
  7. 2024CausalEval: Towards Better Causal Reasoning in Language Models
  8. 2026The Abstraction Gap in Vision-Language Causal Reasoning