# Can Transformers Reason About Effects of Actions?

arXiv: https://arxiv.org/abs/2012.09938 · HTML: https://mlsanity.com/paper/2012.09938/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Dec 2020.

## Built on by (most cited first, 9 of 9)

- [Leveraging Pre-trained Large Language Models to Construct and Utilize World Models for Model-based Task Planning](https://mlsanity.com/paper/2305.14909/) (2023)
- [Towards LogiGLUE: A Brief Survey and A Benchmark for Analyzing Logical Reasoning Capabilities of Language Models](https://mlsanity.com/paper/2310.00836/) (2023)
- [Text2World: Benchmarking Large Language Models for Symbolic World Model Generation](https://mlsanity.com/paper/2502.13092/) (2025)
- ["John is 50 years old, can his son be 65?" Evaluating NLP Models' Understanding of Feasibility](https://mlsanity.com/paper/2210.07471/) (2022)
- [MDDial: A Multi-turn Differential Diagnosis Dialogue Dataset with Reliability Evaluation](https://mlsanity.com/paper/2308.08147/) (2023)
- [Dyna-bAbI: unlocking bAbI's potential with dynamic synthetic benchmarking](https://mlsanity.com/paper/2112.00086/) (2021)
- [ActionReasoningBench: Reasoning about Actions with and without Ramification Constraints](https://mlsanity.com/paper/2406.04046/) (2024)
- [Learning Action-Effect Dynamics for Hypothetical Vision-Language Reasoning Task](https://mlsanity.com/paper/2212.03866/) (2022)
- [Learning Action-Effect Dynamics from Pairs of Scene-graphs](https://mlsanity.com/paper/2212.03433/) (2022)

## Abstract (arXiv)

A recent work has shown that transformers are able to "reason" with facts and rules in a limited setting where the rules are natural language expressions of conjunctions of conditions implying a conclusion. Since this suggests that transformers may be used for reasoning with knowledge given in natural language, we do a rigorous evaluation of this with respect to a common form of knowledge and its corresponding reasoning -- the reasoning about effects of actions. Reasoning about action and change has been a top focus in the knowledge representation subfield of AI from the early days of AI and more recently it has been a highlight aspect in common sense question answering. We consider four action domains (Blocks World, Logistics, Dock-Worker-Robots and a Generic Domain) in natural language and create QA datasets that involve reasoning about the effects of actions in these domains. We investigate the ability of transformers to (a) learn to reason in these domains and (b) transfer that learning from the generic domains to the other domains.