# Causal Reasoning in Simulation for Structure and Transfer Learning of Robot Manipulation Policies

arXiv: https://arxiv.org/abs/2103.16772 · HTML: https://mlsanity.com/paper/2103.16772/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Mar 2021.

## Built on by (most cited first, 22 of 22)

- [Learning Linear Causal Representations from Interventions under General Nonlinear Mixing](https://mlsanity.com/paper/2306.02235/) (2023)
- [Score-based Causal Representation Learning with Interventions](https://mlsanity.com/paper/2301.08230/) (2023)
- [A Survey on Causal Reinforcement Learning](https://mlsanity.com/paper/2302.05209/) (2023)
- [Causal Reasoning Meets Visual Representation Learning: A Prospective Study](https://mlsanity.com/paper/2204.12037/) (2022)
- [Causal Reinforcement Learning: A Survey](https://mlsanity.com/paper/2307.01452/) (2023)
- [Invariant Causal Imitation Learning for Generalizable Policies](https://mlsanity.com/paper/2311.01489/) (2023)
- [Why did I fail? A Causal-based Method to Find Explanations for Robot Failures](https://mlsanity.com/paper/2204.04483/) (2022)
- [Learning by Watching: A Review of Video-based Learning Approaches for Robot Manipulation](https://mlsanity.com/paper/2402.07127/) (2024)
- [Situation-Dependent Causal Influence-Based Cooperative Multi-agent Reinforcement Learning](https://mlsanity.com/paper/2312.09539/) (2023)
- [Assessing the Impact of Distribution Shift on Reinforcement Learning Performance](https://mlsanity.com/paper/2402.03590/) (2024)
- [ROS-Causal: A ROS-based Causal Analysis Framework for Human-Robot Interaction Applications](https://mlsanity.com/paper/2402.16068/) (2024)
- [Identifying Selections for Unsupervised Subtask Discovery](https://mlsanity.com/paper/2410.21616/) (2024)
- [Experimental Evaluation of ROS-Causal in Real-World Human-Robot Spatial Interaction Scenarios](https://mlsanity.com/paper/2406.04955/) (2024)
- [BECAUSE: Bilinear Causal Representation for Generalizable Offline Model-based Reinforcement Learning](https://mlsanity.com/paper/2407.10967/) (2024)
- [Exploring Causality for HRI: A Case Study on Robotic Mental Well-being Coaching](https://mlsanity.com/paper/2503.11684/) (2025)
- [Learning Low-Level Causal Relations using a Simulated Robotic Arm](https://mlsanity.com/paper/2410.07751/) (2024)
- [Causality-enhanced Decision-Making for Autonomous Mobile Robots in Dynamic Environments](https://mlsanity.com/paper/2504.11901/) (2025)
- [Automating Transfer of Robot Task Plans using Functorial Data Migrations](https://mlsanity.com/paper/2406.15961/) (2024)
- [Causality-Based Reinforcement Learning Method for Multi-Stage Robotic Tasks](https://mlsanity.com/paper/2503.03145/) (2025)
- [Creative Robot Tool Use by Counterfactual Reasoning](https://mlsanity.com/paper/2605.05411/) (2026)
- [Interventional Causal Circuits for Safe Robot Action Testing and Failure Recovery](https://mlsanity.com/paper/2607.14826/) (2026)
- [Estimating the Causal Impact of Social Navigation Signals in Corridor Scenarios](https://mlsanity.com/paper/2610.16411/) (2026)

## Abstract (arXiv)

We present CREST, an approach for causal reasoning in simulation to learn the relevant state space for a robot manipulation policy. Our approach conducts interventions using internal models, which are simulations with approximate dynamics and simplified assumptions. These interventions elicit the structure between the state and action spaces, enabling construction of neural network policies with only relevant states as input. These policies are pretrained using the internal model with domain randomization over the relevant states. The policy network weights are then transferred to the target domain (e.g., the real world) for fine tuning. We perform extensive policy transfer experiments in simulation for two representative manipulation tasks: block stacking and crate opening. Our policies are shown to be more robust to domain shifts, more sample efficient to learn, and scale to more complex settings with larger state spaces. We also show improved zero-shot sim-to-real transfer of our policies for the block stacking task.