# Variable Impedance Control in End-Effector Space: An Action Space for Reinforcement Learning in Contact-Rich Tasks

arXiv: https://arxiv.org/abs/1906.08880 · HTML: https://mlsanity.com/paper/1906.08880/

## Key facts

- Cited by 67 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Jun 2019.

## Built on by (most cited first, 30 of 67)

- [Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learning](https://mlsanity.com/paper/2108.10470/) (2021)
- [ManiSkill2: A Unified Benchmark for Generalizable Manipulation Skills](https://mlsanity.com/paper/2302.04659/) (2023)
- [Orbit: A Unified Simulation Framework for Interactive Robot Learning Environments](https://mlsanity.com/paper/2301.04195/) (2023)
- [Deep Reinforcement Learning for Robotics: A Survey of Real-World Successes](https://mlsanity.com/paper/2408.03539/) (2024)
- [BEHAVIOR: Benchmark for Everyday Household Activities in Virtual, Interactive, and Ecological Environments](https://mlsanity.com/paper/2108.03332/) (2021)
- [Structured World Models from Human Videos](https://mlsanity.com/paper/2308.10901/) (2023)
- [Imitating Task and Motion Planning with Visuomotor Transformers](https://mlsanity.com/paper/2305.16309/) (2023)
- [Accelerating Robotic Reinforcement Learning via Parameterized Action Primitives](https://mlsanity.com/paper/2110.15360/) (2021)
- [TRANSIC: Sim-to-Real Policy Transfer by Learning from Online Correction](https://mlsanity.com/paper/2405.10315/) (2024)
- [Factory: Fast Contact for Robotic Assembly](https://mlsanity.com/paper/2205.03532/) (2022)
- [Efficient Reinforcement Learning for Autonomous Driving with Parameterized Skills and Priors](https://mlsanity.com/paper/2305.04412/) (2023)
- [LASER: Learning a Latent Action Space for Efficient Reinforcement Learning](https://mlsanity.com/paper/2103.15793/) (2021)
- [Learning to Grasp the Ungraspable with Emergent Extrinsic Dexterity](https://mlsanity.com/paper/2211.01500/) (2022)
- [On the Role of the Action Space in Robot Manipulation Learning and Sim-to-Real Transfer](https://mlsanity.com/paper/2312.03673/) (2023)
- [GLiDE: Generalizable Quadrupedal Locomotion in Diverse Environments with a Centroidal Model](https://mlsanity.com/paper/2104.09771/) (2021)
- [DriverGym: Democratising Reinforcement Learning for Autonomous Driving](https://mlsanity.com/paper/2111.06889/) (2021)
- [Robust High-speed Running for Quadruped Robots via Deep Reinforcement Learning](https://mlsanity.com/paper/2103.06484/) (2021)
- [Hierarchical Neural Dynamic Policies](https://mlsanity.com/paper/2107.05627/) (2021)
- [InsertionNet -- A Scalable Solution for Insertion](https://mlsanity.com/paper/2104.14223/) (2021)
- [Soft and Rigid Object Grasping With Cross-Structure Hand Using Bilateral Control-Based Imitation Learning](https://mlsanity.com/paper/2311.09555/) (2023)
- [SRL-VIC: A Variable Stiffness-Based Safe Reinforcement Learning for Contact-Rich Robotic Tasks](https://mlsanity.com/paper/2406.13744/) (2024)
- [A Survey on Imitation Learning for Contact-Rich Tasks in Robotics](https://mlsanity.com/paper/2506.13498/) (2025)
- [Robotic Table Tennis: A Case Study into a High Speed Learning System](https://mlsanity.com/paper/2309.03315/) (2023)
- [Learning Variable Impedance Control via Inverse Reinforcement Learning for Force-Related Tasks](https://mlsanity.com/paper/2102.06838/) (2021)
- [Robotic Table Wiping via Reinforcement Learning and Whole-body Trajectory Optimization](https://mlsanity.com/paper/2210.10865/) (2022)
- [FACET: Force-Adaptive Control via Impedance Reference Tracking for Legged Robots](https://mlsanity.com/paper/2505.06883/) (2025)
- [Skill-based Multi-objective Reinforcement Learning of Industrial Robot Tasks with Planning and Knowledge Integration](https://mlsanity.com/paper/2203.10033/) (2022)
- [Learning to Centralize Dual-Arm Assembly](https://mlsanity.com/paper/2110.04003/) (2021)
- [GenCHiP: Generating Robot Policy Code for High-Precision and Contact-Rich Manipulation Tasks](https://mlsanity.com/paper/2404.06645/) (2024)
- [OSCAR: Data-Driven Operational Space Control for Adaptive and Robust Robot Manipulation](https://mlsanity.com/paper/2110.00704/) (2021)

## Abstract (arXiv)

Reinforcement Learning (RL) of contact-rich manipulation tasks has yielded impressive results in recent years. While many studies in RL focus on varying the observation space or reward model, few efforts focused on the choice of action space (e.g. joint or end-effector space, position, velocity, etc.). However, studies in robot motion control indicate that choosing an action space that conforms to the characteristics of the task can simplify exploration and improve robustness to disturbances. This paper studies the effect of different action spaces in deep RL and advocates for Variable Impedance Control in End-effector Space (VICES) as an advantageous action space for constrained and contact-rich tasks. We evaluate multiple action spaces on three prototypical manipulation tasks: Path Following (task with no contact), Door Opening (task with kinematic constraints), and Surface Wiping (task with continuous contact). We show that VICES improves sample efficiency, maintains low energy consumption, and ensures safety across all three experimental setups. Further, RL policies learned with VICES can transfer across different robot models in simulation, and from simulation to real for the same robot. Further information is available at https://stanfordvl.github.io/vices.