# Sample-Efficient Reinforcement Learning with Symmetry-Guided Demonstrations for Robotic Manipulation

arXiv: https://arxiv.org/abs/2304.06055 · HTML: https://mlsanity.com/paper/2304.06055/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 33 references that are in the map.
- First posted on arXiv 12 Apr 2023.

## Builds on (most cited first, 11 of 11)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Overcoming Exploration in Reinforcement Learning with Demonstrations](https://mlsanity.com/paper/1709.10089/) (2017)
- [Revisiting Fundamentals of Experience Replay](https://mlsanity.com/paper/2007.06700/) (2020)
- [Sample-Efficient Reinforcement Learning with Stochastic Ensemble Value Expansion](https://mlsanity.com/paper/1807.01675/) (2018)
- [Reinforcement Learning from Imperfect Demonstrations](https://mlsanity.com/paper/1802.05313/) (2018)
- [Equivariant Q Learning in Spatial Action Spaces](https://mlsanity.com/paper/2110.15443/) (2021)
- [HRL4IN: Hierarchical Reinforcement Learning for Interactive Navigation with Mobile Manipulators](https://mlsanity.com/paper/1910.11432/) (2019)
- [Symmetry Learning for Function Approximation in Reinforcement Learning](https://mlsanity.com/paper/1706.02999/) (2017)
- [Dexterous Robotic Manipulation using Deep Reinforcement Learning and Knowledge Transfer for Complex Sparse Reward-based Tasks](https://mlsanity.com/paper/2205.09683/) (2022)
- [Neural Computing](https://mlsanity.com/paper/2107.02744/) (2021)

## Abstract (arXiv)

Reinforcement learning (RL) suffers from low sample efficiency, particularly in high-dimensional continuous state-action spaces of complex robotic manipulation tasks. RL performance can improve by leveraging prior knowledge, even when demonstrations are limited and collected from simplified environments. To address this, we define General Abstract Symmetry (GAS) for aggregating demonstrations from symmetrical abstract partitions of the robot environment. We introduce Demo-EASE, a novel training framework using a dual-buffer architecture that stores both demonstrations and RL-generated experiences. Demo-EASE improves sample efficiency through symmetry-guided demonstrations and behavior cloning, enabling strong initialization and balanced exploration-exploitation. Demo-EASE is compatible with both on-policy and off-policy RL algorithms, supporting various training regimes. We evaluate our framework in three simulation experiments using a Kinova Gen3 robot with joint-space control within PyBullet. Our results show that Demo-EASE significantly accelerates convergence and improves final performance compared to standard RL baselines, demonstrating its potential for efficient real-world robotic manipulation learning.