# Symmetry-aware Reinforcement Learning for Robotic Assembly under Partial Observability with a Soft Wrist

arXiv: https://arxiv.org/abs/2402.18002 · HTML: https://mlsanity.com/paper/2402.18002/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods.
- Builds on 19 of its 43 references that are in the map.
- First posted on arXiv 28 Feb 2024.

## Placed in

- Deep reinforcement learning methods / [RL for robot locomotion and manipulation](https://mlsanity.com/deeprl/rl-for-robot-locomotion-and-manipulation/)

## Builds on (most cited first, 19 of 19)

- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [robosuite: A Modular Simulation Framework and Benchmark for Robot Learning](https://mlsanity.com/paper/2009.12293/) (2020)
- [General E(2)-Equivariant Steerable CNNs](https://mlsanity.com/paper/1911.08251/) (2019)
- [Making Sense of Vision and Touch: Self-Supervised Learning of Multimodal Representations for Contact-Rich Tasks](https://mlsanity.com/paper/1810.10191/) (2018)
- [MDP Homomorphic Networks: Group Symmetries in Reinforcement Learning](https://mlsanity.com/paper/2006.16908/) (2020)
- [Recurrent Model-Free RL Can Be a Strong Baseline for Many POMDPs](https://mlsanity.com/paper/2110.05038/) (2021)
- [SO(2)-Equivariant Reinforcement Learning](https://mlsanity.com/paper/2203.04439/) (2022)
- [Sample Efficient Grasp Learning Using Equivariant Models](https://mlsanity.com/paper/2202.09468/) (2022)
- [The Surprising Effectiveness of Equivariant Models in Domains with Latent Symmetry](https://mlsanity.com/paper/2211.09231/) (2022)
- [Variable Compliance Control for Robotic Peg-in-Hole Assembly: A Deep Reinforcement Learning Approach](https://mlsanity.com/paper/2008.10224/) (2020)
- [A General Theory of Correct, Incorrect, and Extrinsic Equivariance](https://mlsanity.com/paper/2303.04745/) (2023)
- [Leveraging Fully Observable Policies for Learning under Partial Observability](https://mlsanity.com/paper/2211.01991/) (2022)
- [Compare Contact Model-based Control and Contact Model-free Learning: A Survey of Robotic Peg-in-hole Assembly Strategies](https://mlsanity.com/paper/1904.05240/) (2019)
- [Belief-Grounded Networks for Accelerated Robot Learning under Partial Observability](https://mlsanity.com/paper/2010.09170/) (2020)
- [Nonlinear System Identification of Soft Robot Dynamics Using Koopman Operator Theory](https://mlsanity.com/paper/1810.06637/) (2018)
- [Transferable Force-Torque Dynamics Model for Peg-in-hole Task](https://mlsanity.com/paper/1912.00260/) (2019)
- [Zero-Shot Transfer of Haptics-Based Object Insertion Policies](https://mlsanity.com/paper/2301.12587/) (2023)
- [Sample Efficient Dynamics Learning for Symmetrical Legged Robots:Leveraging Physics Invariance and Geometric Symmetries](https://mlsanity.com/paper/2210.07329/) (2022)
- [Compliant finray-effect gripper for high-speed robotic assembly of electrical components](https://mlsanity.com/paper/2301.08431/) (2023)

## Built on by (most cited first, 9 of 9)

- [Equivariant Diffusion Policy](https://mlsanity.com/paper/2407.01812/) (2024)
- [MATCH POLICY: A Simple Pipeline from Point Cloud Registration to Manipulation Policies](https://mlsanity.com/paper/2409.15517/) (2024)
- [Constraint-Preserving Data Generation for Visuomotor Policy Learning](https://mlsanity.com/paper/2508.03944/) (2025)
- [A Practical Guide for Incorporating Symmetry in Diffusion Policy](https://mlsanity.com/paper/2505.13431/) (2025)
- [Equivariant Ensembles and Regularization for Reinforcement Learning in Map-based Path Planning](https://mlsanity.com/paper/2403.12856/) (2024)
- [VE2VF: Vision-Enabled to Vision-Free Distillation via Real-world Reinforcement Learning for Robust Contact-Rich Manipulation](https://mlsanity.com/paper/2605.29564/) (2026)
- [Pix2Act: Image-Space Manipulation Policies with Equivariant Augmentation](https://mlsanity.com/paper/2607.11167/) (2026)
- [Harnessing human expertise for high-precision robotic assembly in industrialized construction: A sample-efficient installer-in-the-loop interactive reinforcement learning framework](https://mlsanity.com/paper/2609.13234/) (2026)
- [Reinforcement Learning from Real-Robot Interaction without High-Quality State Estimates](https://mlsanity.com/paper/2610.14042/) (2026)

## Abstract (arXiv)

This study tackles the representative yet challenging contact-rich peg-in-hole task of robotic assembly, using a soft wrist that can operate more safely and tolerate lower-frequency control signals than a rigid one. Previous studies often use a fully observable formulation, requiring external setups or estimators for the peg-to-hole pose. In contrast, we use a partially observable formulation and deep reinforcement learning from demonstrations to learn a memory-based agent that acts purely on haptic and proprioceptive signals. Moreover, previous works do not incorporate potential domain symmetry and thus must search for solutions in a bigger space. Instead, we propose to leverage the symmetry for sample efficiency by augmenting the training data and constructing auxiliary losses to force the agent to adhere to the symmetry. Results in simulation with five different symmetric peg shapes show that our proposed agent can be comparable to or even outperform a state-based agent. In particular, the sample efficiency also allows us to learn directly on the real robot within 3 hours.