# RoCoDA: Counterfactual Data Augmentation for Data-Efficient Robot Learning from Demonstrations

arXiv: https://arxiv.org/abs/2411.16959 · HTML: https://mlsanity.com/paper/2411.16959/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Placed in Robot manipulation imitation learning, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 28 of its 31 references that are in the map.
- First posted on arXiv 25 Nov 2024.

## Placed in

- Offline RL and imitation learning from data / [Robot manipulation imitation learning](https://mlsanity.com/offlinerl/robot-manipulation-imitation-learning/)

## Builds on (most cited first, 28 of 28)

- [RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control](https://mlsanity.com/paper/2307.15818/) (2023)
- [Diffusion Policy: Visuomotor Policy Learning via Action Diffusion](https://mlsanity.com/paper/2303.04137/) (2023)
- [RT-1: Robotics Transformer for Real-World Control at Scale](https://mlsanity.com/paper/2212.06817/) (2022)
- [Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware](https://mlsanity.com/paper/2304.13705/) (2023)
- [LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learning](https://mlsanity.com/paper/2306.03310/) (2023)
- [Open X-Embodiment: Robotic Learning Datasets and RT-X Models](https://mlsanity.com/paper/2310.08864/) (2023)
- [DROID: A Large-Scale In-The-Wild Robot Manipulation Dataset](https://mlsanity.com/paper/2403.12945/) (2024)
- [Solving Rubik's Cube with a Robot Hand](https://mlsanity.com/paper/1910.07113/) (2019)
- [What Matters in Learning from Offline Human Demonstrations for Robot Manipulation](https://mlsanity.com/paper/2108.03298/) (2021)
- [BridgeData V2: A Dataset for Robot Learning at Scale](https://mlsanity.com/paper/2308.12952/) (2023)
- [Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixels](https://mlsanity.com/paper/2004.13649/) (2020)
- [Perceiver-Actor: A Multi-Task Transformer for Robotic Manipulation](https://mlsanity.com/paper/2209.05451/) (2022)
- [Reinforcement Learning with Augmented Data](https://mlsanity.com/paper/2004.14990/) (2020)
- [MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrations](https://mlsanity.com/paper/2310.17596/) (2023)
- [RVT: Robotic View Transformer for 3D Object Manipulation](https://mlsanity.com/paper/2306.14896/) (2023)
- [Scaling Robot Learning with Semantically Imagined Experience](https://mlsanity.com/paper/2302.11550/) (2023)
- [RoboAgent: Generalization and Efficiency in Robot Manipulation via Semantic Augmentations and Action Chunking](https://mlsanity.com/paper/2309.01918/) (2023)
- [Behavior Generation with Latent Actions](https://mlsanity.com/paper/2403.03181/) (2024)
- [Visual Imitation Made Easy](https://mlsanity.com/paper/2008.04899/) (2020)
- [GenAug: Retargeting behaviors to unseen situations via Generative Augmentation](https://mlsanity.com/paper/2302.06671/) (2023)
- [Synthetic Experience Replay](https://mlsanity.com/paper/2303.06614/) (2023)
- [Counterfactual Data Augmentation using Locally Factored Dynamics](https://mlsanity.com/paper/2007.02863/) (2020)
- [S4RL: Surprisingly Simple Self-Supervision for Offline Reinforcement Learning](https://mlsanity.com/paper/2103.06326/) (2021)
- [Sample-Efficient Reinforcement Learning via Counterfactual-Based Data Augmentation](https://mlsanity.com/paper/2012.09092/) (2020)
- [QueST: Self-Supervised Skill Abstractions for Learning Continuous Control](https://mlsanity.com/paper/2407.15840/) (2024)
- [MoCoDA: Model-based Counterfactual Data Augmentation](https://mlsanity.com/paper/2210.11287/) (2022)
- [Revisiting the Adversarial Robustness-Accuracy Tradeoff in Robot Learning](https://mlsanity.com/paper/2204.07373/) (2022)
- [Self-Supervised Learning of Action Affordances as Interaction Modes](https://mlsanity.com/paper/2305.17565/) (2023)

## Built on by (most cited first, 13 of 13)

- [AirExo-2: Scaling up Generalizable Robotic Imitation Learning with Low-Cost Exoskeletons](https://mlsanity.com/paper/2503.03081/) (2025)
- [Constraint-Preserving Data Generation for Visuomotor Policy Learning](https://mlsanity.com/paper/2508.03944/) (2025)
- [See Once, Then Act: Vision-Language-Action Model with Task Learning from One-Shot Video Demonstrations](https://mlsanity.com/paper/2512.07582/) (2025)
- [Prepare Before You Act: Learning From Humans to Rearrange Initial States](https://mlsanity.com/paper/2509.18043/) (2025)
- [DynamicManip: Enabling Dynamic Manipulation from a Single Static Demonstration](https://mlsanity.com/paper/2608.01452/) (2026)
- [GuidedAttention: Interpretable and Correctable Visual Attention for OOD-Robust Robot Manipulation via Imitation Learning](https://mlsanity.com/paper/2607.21049/) (2026)
- [Iterative Compositional Data Generation for Robot Control](https://mlsanity.com/paper/2512.10891/) (2025)
- [MinInter: Minimizing Trajectory Interpolation During Data Augmentation for Imitation Learning](https://mlsanity.com/paper/2606.24078/) (2026)
- [Robustness of Robotic Manipulation: Foundations and Frontiers](https://mlsanity.com/paper/2606.31494/) (2026)
- [Vision-Language Grounded Task-Context-Aware Imitation Learning for Robotic Disassembly](https://mlsanity.com/paper/2609.17714/) (2026)
- [CAPEX: Efficiently Distilling Foundation Model Behavior into Deployable Robot Policies through Experience-Adaptive Reasoning](https://mlsanity.com/paper/2609.33007/) (2026)
- [When Instructions Retrieve Trajectories: Diagnosing and Mitigating Generalization Failures in VLA Models](https://mlsanity.com/paper/2609.39971/) (2026)
- [PerturBot: Breaking Shortcut Priors in Vision-Language-Action Models with Perturbative Training](https://mlsanity.com/paper/2610.04616/) (2026)

## Abstract (arXiv)

Imitation learning in robotics faces significant challenges in generalization due to the complexity of robotic environments and the high cost of data collection. We introduce RoCoDA, a novel method that unifies the concepts of invariance, equivariance, and causality within a single framework to enhance data augmentation for imitation learning. RoCoDA leverages causal invariance by modifying task-irrelevant subsets of the environment state without affecting the policy's output. Simultaneously, we exploit SE(3) equivariance by applying rigid body transformations to object poses and adjusting corresponding actions to generate synthetic demonstrations. We validate RoCoDA through extensive experiments on five robotic manipulation tasks, demonstrating improvements in policy performance, generalization, and sample efficiency compared to state-of-the-art data augmentation methods. Our policies exhibit robust generalization to unseen object poses, textures, and the presence of distractors. Furthermore, we observe emergent behavior such as re-grasping, indicating policies trained with RoCoDA possess a deeper understanding of task dynamics. By leveraging invariance, equivariance, and causality, RoCoDA provides a principled approach to data augmentation in imitation learning, bridging the gap between geometric symmetries and causal reasoning. Project Page: https://rocoda.github.io